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ABSTRACT 

The Comprehensive School Mathematics Program (CSMP) 
is a program of CEMREL, Inc., one of the national educational 
laboratories, and is funded by the 'National Institute of Education. 
Its major purpose is the development of curriculum materials in 
mathematics for grades K-6. Beginning in September, 1973, CSMP began 
an extended pilot trial of its Elementary Program. During the second 
year, 1974-75, thirteen classes continued use of CSMP in the second 
grade. In May and June, 1975, another series of tests was 
administered to the second graders and to their comparison classes. 
This report describes these tests and the resulting data. All 
differences that were significant favored CSMP. Results were 
consistent across school districts and ability levels. Some learning 
problems with the materials are identified. (RH) 
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Introduction 

In the fall of 1973, the Comprehensive School Mathematics Program (CSMP) 
began a longitudinal pilot study of its Elementary Program. Over 100 teachers 
began using the program, either in first grade or kindergarten. During the 
1974-75 school year just past, the second year of this pilot study, most of 
these classes continued into second grade and first grade respectively and 
many new classes began using CSMP materials. 

For the purposes of the pilot study, classes in the St. Louis area are 
designated "local". For these classes teacher training is standardized and 
comparison classes established. These local classes provide much of the 
evaluation data derived from the pilot study including data related to class- 
room observations, student and teacher interviews and individualized testing. 
Classes not in the St. Louis area are designated "outer ring". These classes 
provide information concerning usage of materials (via questionnaires and 
teacher logs) and various corroborative test data from cooperating sites. 

In the first year of the pilot study, 1973-74, a rather extensive series 
of tests was administered to local first grade CSMP classes and to their paired 
comparison classes.* During the second year of the pilot study just passed, 
1974-75, thirteen of these classes continued as reasonably intact groups to 
study CSMP in the second grade. In May and June another series of tests was. 
administered to these now second graders and to their comparison classes, which 
were, with one exception, the same groups of students used for comparison in the 
first year of the pilot study. This report describes these tests and the resulting 
data^ 
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Content, Evaluation Report l-B-3: End-of-Year Test Data: Standard First 
Grade Content , Evaluation Report l-B-5: Test Data on Some General Cognitive 
Skills Related to CSMP Content 
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Flr_st Year Results and Second Year Setting 

First: Year Results 

In the first -'ear of the Extended Pilot Study, 1973-7A, the focus was on 
the kindergarten and first grade classes in the local St. Louis area, particularly 
the first grade classes. While much data dealing with the implementation of the 
program was collected including teacher logs, auestionnaires, classroom cbservatior 
cost figures, etc. and while much of the above, in addition to some achievement 
data, was collected from "outer ring" (distant from St. Louis) sites, these are 
summarized in full elsewhere* and will not be reviewed here. For the purpose of 
this report, what will be reviewed here is the design and result of testing 
carried out with local fi\st grade classes. The reader who is familiar with these 
results may wish to proceed directly to the next section of this chapter (p. 6). 

There were 16 local first grade classes located in five school districts as 
shown in Table 1. 

Table 1 

Description of Local 
First Grade CSMP Classes, 1973-74 





Class Number 


Mean Number of 
StudenUs/Class 


Predominant 
Racial Makeup 


Est imated 
Economic 


Socio- 
Status 


Type of 
Community 


Black 


1 — — 
White 


Mixed 


Low 


Low-Middle 


Middle 


High Middle 


High 


District 1 


1-4 


32 


/ 




/ 


/ 


/ 








Urban 


District 2 


5-8 


22 






/ 




/ 




/ 




Suburban 


District 3 


9-10 


24 




/ 








/ 






Suburban 


District 4 


11-15 


25 




/ 






/ 

\ 


/ 

V 






Rural/ Suburban 


District 5 


16 


26 




/ 






/ 


/ 






Small Town 



^Evaluation Report: 1-^A-l f Overview, besign and Instrumentation 
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For each of these 16 CSMP classes, a comparison class was established, in 
the same school if possible or in an aOjaceal school if the CSMP class was the 
only first grade class in a school (4 cases). In the fall, the Cognitive Abilities 
Test (Houghton Mifflin, 1962) was adr..inistered to all 32 classes and served as a 
covariate for adjusting end-of-year ivst scores. 

Three kinds of tests were ad-^i uistered in late spring 1974 by specially 
trained testers. Each of r>>v- tes'. is tuore fully described in the referenced 
evaluation report. 

a) Tests of Standar^i Content (Evaluation Report l-B-3) : An analysis 
was made of f fve of the largest selling commercial math series in 
order to determine what skills and concepts are generally taught 
to firsl. 5rade students. Emphasis was placed on the actual tasks 
provided for students, usually in the form of practice exercises. 
Based on this analysis, 8 test scales were constructed, each one 
covering a different content area and generally using the kind of 
item format commonly found in the text books. For each local first 
grade CSMP and comparison class, a random half of the students took 
part of the test; the other half of each class took the rest of the 
test . 

For each test scale, the class mean was calculated for each of the 
16 CSMP and cOi;iparison classes. Table 2 shows the resulting data 
using an analysis of covariance procedure which adjusted scores 
for differences in entering ability as measured by the pretest. 
It can be seen that CSMP classes had significantly (p<,05) higher 
scores on the Larger Number scales and the difference approached 
significance on two other scales. Place Value and Measurement, 
The only scale on which comparison classes had higher scores was 
the Subtraction scale and the difference was not significant. 

Table 2 

Analyses of Class Means 
for Tests of Star^dard Content 



Subtest 


Nu~b'_' r 

of 1 tffii 


CSr.,-^ Non-CS."P 


P lc5-> th-ir. 


Mumera t i on 

Subtraction 

Order 


12 
12 

8 


9-8 9.5 
7. '4 8.'* 
5-6 s.ii 


.22 
.52 


Appi i cat ions 
Larger Numbers (A) 
LcTrqer Numbers (B) 


7 
7 
li 


5.0 5.0 
3.0 2.3 

0.9 OA 


.97 
.01 

.0) 


Place Value 
Measurement 
Add i t i on 


7 

n 


3.'t 2.8 
8.5 7.8 
10.2 9.9 


.05 
.07 
.53 



Adjusted for entering obility base-d on prettst 
I and I^* degrees of freedom 
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b) Tests of Cognitive Skills (Evaluation Report l-B-5) : Three teets 
were constructed which dealt with situations novel to first grade 
students but which were thought to be conceptually related to 
some of the major ideas in the CSMP progriun. The tests dealt with 
classification skills, relational concepts, and analytic thinking 
respectively. Each test was designed to be administered on an 
individual basis, requiring 10-15 minutes each. For each test, all 
the students in from four to six CSMP classes and in the appropriate 
comparison classes were administered the test. 

The analysis of class means, with adjustment for pretest scores, 
was carried out in the usual way. The results are given in 
Table 3. The differences were not significant on any of the 
tests, although CSMP classes had higher mean scores in each case, 
particularly the Relational Thinking and Analytic Reasoning tests. 



Table 3 

Analysis of Class Means 
for Three Tests of Cognitive Skills 



Test 


Niimbe r of 1 terns 


AdiusteH Mean Scores* 
CSM? Mon-CSMP 


F Ratio:** 
P less t\^c)^ 


Class 1 fi cat Ion 


20 


1^.3^ 13^78 




Rcla t lonal 
Thinking - 


15 


10.^2 10.19 


.13 


Analyt ic 
Reason inq 


9 


3.25 2.75 


.16 



* Adjusted for pretest scores 

** 1 and 5> 1 and S> Oiiu 1 cnJ 7 degrees of freedom respectively 



c) Tests of CSMP First Grade Content (l-B-2): A series of 12 subtests, 
comprised of 121 ite^is, was administered to the 16 local CSMP 
classes. The tests were based on the CSMP Content Resvime, a 
docximent prepared by the evaluation staff which outlines the 
development of content based on the tasks contained in the work- 
book series. Content on which student.s had already shown a high 
degree of proficiency in the Mid-Year Test was not retested. 
Because of the spiral nature of the CSMP curriculum, the test 
items of any given content strand exhibited a wide range of 
difficulty levels, ranging from very easy items on which virtually 
all students were expected to be at mastery level, to items testing 
content much further along in the strand which students had very 
little experience with and which only the better ones might get. 
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A very brief summary of the results is given in Table A, The 
ratings given by the evaluation staff for each content area were 
subjective in nature and took into account, besides the percentages 
correct, the number of lessons and amount of workbook practice 
devoted to that content, the level of difficulty of the test items 
and the importance of that content area in preparing for new materials 
It was deemed inadvisable to administer this test to comparison 
classes rot only because of the unique languages (arrows, strings, 
minicomputer) of CSMP but because, based on the analysis described 
above, certain content areas (multiplication, integers, probability) 
are not taught in traditional first grade curricula. 



Table 4 



Summary Data from End-of-Year 
Test of CSMP First Grade Content 



Subtest 


Number 


Mean % 


Rating of 




of Items 


Correct 


Performance* 


Multiplication 


10 


5A 


A - 


li.tegers 


8 


66 


A 


Rationals (a) 


11 


82 


VG 


Rationals (b) 


5 


A9 


A 


Counting Money 


5 


62 


A 


Minicomputer 


17 


51 


A 


Relations 


27 


60 


A 


Addition 


15 


77 


A 


Subtraction 


13 


81 


VG 


Order 




6A 


A 


Probability 


3 


5A 


A 


Venn Diagrams 


1 


39 





*VG-Very Good, A-Adequate, i-Inadequate, VP-Very Poor 
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Second Year Setting 

There were many new kindergarten and first grade classes started locally in 
1974-75 but the evaluation emphasis was on the second grade classes of students 
who would be completing their second year of the CSMP curriculum. After the 
first year of the Extended Pilot Study, schools; made the decision of whether 
or not to continue using CSMP in second grade with the orignial class of 
first graders. One school elected not to continue and in a second the intended 
teacher was transferred to another school too late to arrange training for her 
replacement. The other lA of the sixteen first grade classes were continued 
with second grade CSMP materials. 

In one of the 14 classes more than half the second grade students did 
not have a CSMP background and certain instructional materials were late in 
arriving, prompting the teacher to discontinue the implementation of the program. 

The design was essentially the same as for the previous year. The comparison 
classes established the previous year were, with two exceptions, continued 
virtually intact as second grade comparison classes. In those two cases, new com- 
parison classes were established. The Kuhlmann-Anderson Test, a test of mental 
ability, was administered to all classes in the fall, and these scores were used 
as covariates for adjusted scores on the basis of differing entering ability. This 
test will be referred to subsequently as the "pretest", though not in the sense of 
a pre-post test designed to measure change. As will be seen later such adjust- 
ments were very small because of the excellent matches between the CSMP classes 
and their paired comparison classes. Table 5 summarizes the situation. The 
number of students is the number present for the pretest. Some students were 
absent; others entered or left the class during the school year. But this 
generally happened in about equal nunbers between CSMP and non-CSMP classes so 
that for comparative purposes Table 5 reflects the rather similar class sizes 
and mean ability scores. In the four cases where the comparison class was not 
located in the same school as the CSMP school (because the CSMP class was the 
only second grade class in the school), the comparison class was drawn from an 
adjacent school. 

Table 5 

Local Second Grade Classes 
in Extended Pilot Study, 197A-75 







CSMF Class 


Cc«p«rl«on CXmrnm 






OUcrlct 


Clsis 


lUan Pr«t«st 


Ktj^sr of 


Mmu) fvt—t 


lH«b«r of 


Clsss«s 


Located 


Mviibar 


Mv«Lb«r 


Scor* 




tM Scor* 


Scudants . 


In S«M 


Sohool? 


t 


i 


51.5 


i; 


49.6 


17 


Yes 






2 


52.4 


24 


51.7 


23 


Yes 






'3 * 


62.3 


25 


59.6 


25 


Yss 






4 


42.5 


21 


49.7 


19 


Yes 




2 




52.4 


23 


49.1 


20 


Yes 






6 


56.5 


22 


55.2 


25 


Yss 






7 


58.9 


15 


58.2 


28 


No 






B 


54.2 


25 


53.5 


24 


Yea 




3 


9 


54.3 


28 


49.4 


25 


Yes 






10 


52.5 


23 


49.7 


27 


Yes 






11 


36.1 


27 


31.9 


28 


No 




4 


12 


42.3 


27 


45.0 


26 


No 






13 


45.0 


25 


50.4 


30 


No 








50.8 

1 


23.2 


50.2 


24.4 





*CUss fftit 3 end class pair 4 vera located in the ease school. So trera 
claaa palfi 5 and 6* 

o 1 .5 
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Testing Plan 



Testing Objectives 

1. To compare CSMP and IJon-CSMP students' computational skills and under- 
standing of the content generally considered to be standard in second 
grade mathematics. For this purpose, a standardized test, the 
Comprehensive Test of Basic Skills, Level C, (California Test Bureau, 
1973), was selected and adniinis tered to all second grade classes 

2. To determine whether or not CSMP students were better able to solve 

a wide variety of mathematical "problems", not directly similar to the 
contenc of CGbfP, but related in a transfer or application sense. A 
series of ;.asks was constructed, some of which were administered on an 
individual student basis and some to the whole class. These Comparison 
Tasks v/ere grouped into four tests: Individual Test 1, Individual Test 2, 
Group Test 1 and Group Test 2. These were administered to various CSMP 
and comparison classes according to the plan given below. 

3. To investigate student achievement in the unique aspects of CSMP. A 
series of five tegts^was constructed dealing with specific CSMP content, 
such as the Minicomputer and arrow diagrams. Areas of content already 
tested in 1, above, or indirectly in 2, above, were omitted. These CSMP 
Tests were administered to CSMP classes only, according to the plan below. 

These three groups of tests and the results for them are described in 
succeeding chapters. 

Testing Plan 

Because of classroom time demands it was not possible to administer all 
tests to all classes. The tests were allocated as described in Table 6, below. 
Each test was administered to at least six pairs of classes (CSMP - Non-CSMP) and 
the sample was drawn to ensure representative classes in terms of student ability 
and geographic area. 

For the standardized and comparison tests, the X in Table 6 indicates that 
the test was given to both the CSMP class and its comparison class. In the case 
of the individually administered tests a sample of seven students was chosen 
from each class by the following method. Students were rank ordered on pretest 
scores and selected in such a way as to provide equal intervals of students 
between those selected. For example in a class of 28 students this might mean 
the students with rank 2, 6, 10, lA, 18, 22 and 26 would be selected f or • 
Individual Test 1. Not all selected students took the test because of absentees 
and transfers and alternates were designated when feasible. A different group 
of seven students uas then selected for Individual Test 2. 

Three testers were specially trained to administer the tests. One handled all 
the individual testing, the other two did the group testing. The training 
required about two to three days made up of several short sessions and the 
tests were administered during the last seven weeks of the school year. Four 
class periods of from 25~A5 minutes were required for each class (three for 
Non-CSMP classes since they were not given the CSMP tests). The individual 
tests generally required about 20 minutes per student. 
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Table 6 



Testing Plan for End-of-Year Tests 



District 


CXass 


Standardized Teat 


Comparlaoa Taiti 


C8MP Tiiti 




Pair 


Coaputation 


Concept! and 


Group 


Group 


Individual 


Individual 




B r D F 

O \^ U tm 








Applications 


Test 1 


Teat 2 


Teat 1 


Teit 2 






1 


1 


X 


X 




X 


X 


X 


X 


X X 




2 


X 


X 


X 




X 


Y 
A 




Y Y 
A A 




3 


X 


X 




X 


X 


X 




X X 




U 


X 


X 


X 




X 


X 




X XX 


2 


5 


X 


X 


X 




X 




X 


X 




6 


X 


X 




X 


X 


X 




XX X 




7 


X 


X 


X 




X 


X 


X 


X X 




8 


X 


X 




X 


X 


X 


X 


X X 


3 


9 


X 


X 




X 


X 


X 




X XX 




10 


X 


X 


X 




X 


X 




X X 




11 


X 


X 


X 










X X 


U 


12 


X 


X 




X 


X 


X 


X 


X X 




13 


X 


X 




X 


X 


X 




XX X 


5* 


U 


X 


X 










X 


X X 




15 


X 


X 










X 


X 




16 


X 


X 










X 


X X 



*The three classes in this district were the only three CSMP classes and it was not 
possible to establish adequate conparison classas. Hence thsy are not tackaiically 
pare of the design (comparison) although scores will be raportad saparatsly for thaa 
where appropriate l»tei in the report. 



Method of Analysis 

Two basic procedures were used. When making statistical comparisons between 
CSMP and Non-CSMP classes the class means were treated as the unit of analysis and 
a two-way Analysis of Covariance procedure was used with pretest scores being the 
covariate so that scores wCiTe adjusted for differences in entering ability. One 
classification had two levels; CSMP and non-CSMP. The other classification had 
as many levels as there were class pairings and was essentially a school classifies 
cation. 

Under this model one assumes that the variation in scores can be explained by 
the covariate (difference in entering ability), class pairing (differences in 
school effects), treatment (differences in math curriculum) plus unexplained error. 
The ratio of the variation from treatment differences to the variation from error is 
then the basis of an F test. One hypothesizes that there are no resulting differences 
due to treatment (i.e. between CSMP and Non-CSMP classes) and thaV. resulting 
differences which actually did occur, occurred by chance. One can then determine 
from the F statistic the p-value or probability of this chance occurrence. If the 
probability is low (.05 is often taken as a standard), between CSMP and Non-CSMP 
classes and say that there are '*signif icant" differences. It is this p-value 
which will be reported for each test. 

For this analysis it is necessary that only students who were present for the 
pretest be included in the analysis. Thus, for any particular class, the pretest 
score fo£ a given test will vary slightly depending on which students were absent 
for that test. Later in this report the performance of new-to-CSMP students, 
those who transferntd into a CSMP class in second grade, will be compared with 
students who were orisi^^sl CSMP first graders. 
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Classes designated as Non-CSMP classes were simply that - not using the CSMP 
curriculuEL. These classes used one of three popular elementary mathematics texts 
published by Houghton Mifflin, Addison Wesley and Silver Burdett, but for the 
purpose of this study they have been lumped together as comparison classes as if 
they were studying the same curriculum. In fact these texts are very similar 
(compared to CSMP) in relative emphasis given to various topics (see Evaluation 
Report l-B-3) and one would expect that among classes using these curricula there 
would be only small differences in achievement which could be attributed to a 
particular text . 

For each trc-r .r.ri item a:,,. lysis was produced. Students across all CSMP classes 
were grouped togiitaer and various item and test statistics were derived. The same 
procedure was followed for Non-CSMP students. This enables one to compare the two 
groups on individual test items and see which items or groups cf items were answered 
more successfully by one group or the other. What are provided in the item analyses 
section are thus merely descriptive data; no tests of significance aie made. The 
statistical hypotheses regarding CSML versus Non-CSMP performance are tested through 
analysis of class means (described in the previous section). Thus the item analysis 
supplements and may help in the interpretation of those results. Often in tliis report 
the item analysis precedes the analysis of class means. The reason for this is that 
the item analysis sections provide the actual test items and it seems useful for the 
reader to be able to look at the test items and determine what the test is all about 
before going to the statistical comparison of class means. 



ERIC 
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S^anda^di^ed Tests 

Description of Test 

The Comprehensive Test of Basic Skills is a series of 10 standardized 
achievement tests. Level C is the grade range 1.6 to 2.9 and was standardized 
in 1973. The test is published by the California Test Bureau/McGraw Hill. 

From the Administrator's Manual, Part 1, Description of CTBS: 
"RATIONALE 

A carefully formulated rationale formed che basis for each step in the 
development of all levels of CTBS, Expanded Edition. This rationale 
required that the tests measure systematically those skills prerequisite 
to studying and learning in school. CTBS, Expanded Edition is not intende( 
to measure achievement in specific course content as reflected in text- 
books for various grade levels. The tests are intended for national use, 
by students who have been taught according to various approaches. Test 
items should be answered as readily by students taught according to a 
traditional approach as by those who are taught according to any of the 
newer approaches. However, performance on these tests necessarily depends 
on the possession of relevant knowledge and is affected by the grade level 
at which a skill is first introduced. It is assumed that all curricula 
are formulated to increase, through the grades, a student's competence in 
dealing with content of increasing difficulty. These tests aim to measure 
therefore, those skills common to all curricula." 

and later: 

"Test 7 - Mathematics Computation 

Tlie 28 items in Test 7 consist ot 10 addition, 10 subtraction, and 8 
multiplication problems. Each operation is tested in a separately 
timed section so that a measure of ability for even the slow students 
is provided for all operations. However, there are no separate norms; 
a total score for the three sections will be reported. Addition and 
subtraction problems are displayed in both horizontal and vertical 
formats. They include operations on one-, two-, and three-digit 
numerals. Multiplication problems are in the horizontal format, and 
all are single-digit multiplications. 

Test 8 - Mathematici:: Concepts and Applications 

The 25 items in Test 8 measure skills in basic operations: numbers, 
numeration, measurement, and fractions. The problems are read aloud 
to the students, who select their answers from pictured, numerical, 
or printed responses. One total score is reported for this test."' 



The Computation Tost requires exactly 3A minutes and consists of 28 items and 
tlio Concepts and Applications Test requires approximately 25 minutes and consists 
of 25 items. 
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Analysis of Class Means 

Table 7 gives the class means for each of the three operations and for the 
total of the Computation Test, for the total of the Concepts and Applications 
Te8t and for the Total Mathematics Score, Also given is the mean class pretest 
scores, which may vary somewhat for a given class because of student absentees 
(the tests were administered at different times). 

Table 7 
Class Mean Scores 



Comprehensive Test of Basic Skills 



Class 
Fair 


Compucntion Test 


Concepts <jnd 
Appllca^: lon^ 


TolaJ 
Score 


^'ret est 






-.all" 


Tot.'il 


I'rct:c;;t 


C. ..nu A. 


Prt'tLst; 


1 


7.88 
6.87 


5.53 


5.00 
5.5J 


19.32 
17.93 


50.9 
51.1 


18.13 
17.60 


50.9 
51 . I 


37. A5 
35.53 


50.9 
51.1 


2 


3.09 


6.A1 
6. 22 


6. A 5 
6 . V 1 


20.')'. 
2t/.87 


S2.A 
51 .7 


19. OA 
18.96 


52. A 
51 . 7 


39.95 
39. B 3 


52. A 
51 .7 


3 


8.76 
7. 76 


9. 71 
8. 72 


7.33 
7.A8 


25.80 
23.96 


61.1 
60. 5 


23.1 A 
22. Ao 


61.1 
6J .1 


A8.95 
A 0.66 


01. 1 
61.] 


A 


C . AO 
6.5)6 


6. 1 3 
5.3P 


6 . 0 7 
• 5.50 


18.60 
I 7. A A 


A 2 . A 
50.8 


1 7 . 3f: 


A 3 . 5 
50.8 


3A.73 
3A.8? 


A 7. . A 
50.8 


5 


8. 26 
7.07 


S. 00 


6.05 
7.13 


^2 . 31 
22.07 


52.1 

:.?.3 


1 ^* . 

19. OA 


S9 1 
51 .8 


A] .57 

A") .88 


52.1 
52.3 


O 


9.00 
6.A9 


8.71 
8.81 


6.«)A 
6.? A 


2 A. 6 5 
23.53 

2A.A(, 

?! sr. 


55.7 
57. A 

60.2 
58.8 


21.00 
20.36 


55.7 
56.9 


A5.65 
AA.3A 


55.7 
5 7. A 


7 


8.23 
7. 30 


8. 77 
7. JV 


r.. no 


20. A6 
1 K.^S 

21 J^'y 
20.25 

21.21 
16. R6 


60.2 

58. R 

55.1 
5A.3 


A A. 92 
AO.Al 
_ . 

A A. 90 
A2.21 


60.2 
58.8 

55.1 
5A.6 


8 


8.21 
7.25 


8.25 
8. AO 


6.6/ 
6.00 


23. 13 

21.65 


5 A. A 
5A.I 


9 


7.92 
6.72 


8. 38 
6.5f^ 


7.38 
6.77 


23.68 
20.08 


5A.2 
A9.0 


53.6 
A8.6 


AA.70 
36.81 


52.8 
A8.6 


10 


7.26 
6. AO 


6.70 
7.00 


6.00 
5.9? 


19.76 
19.3:1 


52.5 
50.0 


1 7.83 
15.60 


52.5 
50.0 


37.50 
3 A. 9 2 


S2.5 
50.0 


11 


6.00 
5.A2 


5.33 
A. 33 


5.06 
3.75 


16.3^1 
1 'j.SO 


36.9 
31.9 


15.7C 
I 0 . 

16.65 
1 7.82 


36.9 
32.2 

A 2. 6 
A A. A 


32.1 7 
2A.52 

37. A3 
36.^^.6 


36.9 
32.2 


12 


6.92 
6.A5 




6.05 


20. A \\ 
in.lA 


A2.7 
A A . A 


A 2 . 6 
A A. A 

A 7.0 
50. A 


13 


7.57 
7.68 


7.05 
8.1A 


7.03 
6.82 


21.67 
22. 6A 


A7.0 
50. A 


16.29 
18.61 


A7.0 
50. A 


37.96 
A1.25 
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Table 8, below, gives the mean score across the 13 CSMP classes and across 
the 13 Non-CSMP classes for the various tests of the Comprehensive Tests of 
Basic Skills. Also given for the two groups are the mean pretest scores vjhich 
are almost equal. The adjustment in scores due to differences in entering 
ability never exceeded .05 (though the pretest was effective in reducing the 
unexplained error variance; from 30% to 75% on the two tests and total score). 
Other traditional Analysis of Covariance statistics are given in the Appendix 
for this and subsequent series of tests. Because of the excellent match of 
classes the adjustments in scores due to differences in entering ability 
pretest scores were small, never exceeding .05 raw score units. 

Table 8 



Mean Scores and Significance Tests 
CTBS: CSMP versus Non-CSMP 



Test 


Mean Test Score 
Across Classes 


Mean Pretest Score 
Across Classes 


F-Test, 1 and U 
degrees of freedom 
p less than: 


CSMP 


Non-CSMP 


CSMP 


Non-CSMP 


Computation Test 


21.63 


20.28 








.01 


Addition 


7.8 


7,1 




50.95 


50.95 


• 01 


Subtraction 


7,5 


7,0 








• 05 


Multiplication 


6.4 


6.2 








.28 


Concepts and Applications Test 


18.96 


18.12 




51.03 


50,92 


.03 


Total Mathematics Score 


40,61 


38.47 




50.88 


51.03 


.01 



The p value given in the right hand column of Table 8 is based on the assumption 
that there are no real differences between CSMP and non-CSMP classes and that the 
actual observed differences occurred by "chance". The p value is the probability 
of this charce event. It can be seen that CSMP classes scpred significantly 
hi^^her than Non-CSMP classes on both the Computations Test and the* Concepts and 
Applications Test, as well as on the Total Mathematics score. On the three 
parts of the Computations Test the differences were significant for the addition 
and for the subtraction items, but not for the multiplications items. 

Item Analysis 

Table 9, below, gives the percent correct for each iten. in the Computation 
for CSMP students and for Non-CSMP students. Unlike the previous section, data 
are averaged across students, not across classes:, ant^ therefore the data do not 
agree exactly with Table 8. Table 10 provides the same infom.'itlon for the 
Concepts and Applications Test. For brevity the optional answers to these 
multiple choice tests are not usually given and in Table 10, the oral directions 
accompanying each item have hetm shortened somewhat. 
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Percent Correct 
Computation Test 



Addition Items 



It! 



2. 



3 
8 
0 

+ 6 

i9 
6 

+128 



4. 27 
♦18 

5. $1.14 

.63 

6. 13 

6 
52 
+ 27 



/. AO+30- 

8. 7+3+2- 

9. 11+7- 



10. 26+21- 



CSMP 
Students 



88 



79 



74 



63 



50 



60 



91 
94 
93 



79 



Non-CSHP 
Students 



88 



81 



64 



55 



39 



49 



86 
93 
90 



73 



subtraction Items 



Itea 



11. 



12. 



13. 



38 

- b 

545 
- Jj 

16 

- 8 



14. 



15. 



16. 



768 
-427 

55 
'52 

64 
- 9 



17. 65-42- 



18. 24-8- 



19. 13-8- 



20. 78-43- 





CSMP 


Non-CSMP 


NuBber of Students 


258 


283 


Hsjin Score 


21.70 


20.65 


Standard Deviation 


5.89 


5.63 


Ksan Prstsst Score 


50.02 


50.37 


Correlation with Pretest 


.65 


.57 



CSMP 
Students 



88 



74 



75 



79 



70 



68 



83 



76 



69 



68 



Non-CSHP 
Students 



82 



72 



73 



70 



69 



61 



81 



71 



70 



63 



Multiplication Items 



Item 



21. 5x2- 



22. 1x4- 



!3. 2x3- 



24 , 3x5- 



25. 3x4- 



26. 5x9- 



27. 4x4- 



28. 8x5- 



CSMP 
Students 



87 



86 



86 



83 



83 



72 



72 



78 



Non-CSMP 
Students 



88 



84 



86 



86 



81 



70 



69 



70 



There are no single items on which there are large differences between the 
two groups of students; in fact the largest difference in percent correct is 
only 11 (items 5,6). However, the differences were consistently in favor of 
CSMP students. On only four items did the Non-CSMP students have a liigher 
percent correct (and then by at most 3%), even though they had a slightly higher 
pretest score. 
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Table 10 



Percent Correct 
Concepts and ApplicatioriB Test 









CSMP 


nOn-LbPlr 




Item 




Students 


S tudents 




Away of showing 13: 11+2, 12+2, 11 + 1, lCH-2 


Oj 


O J 


2. 


In which box does number word show how many boats there are: 






3. 


(4 picture-word pairs shown) 

Two hundreds, three tens and six ones: 236, 263, 326, 362 


78 
78 


75 
81 


4. 


Which shows counting by fives starting at 25? 


8o 




5. 


Which shows counting by twos starting at 


77? 


69 


62 


6. 


Sold by the quart: pictures of bread, eggs, milk, oranges 


73 


76 


7. 


Amount of money shows: (picture of quarter, dime, penny) 


67 


74 


8. 


Same amount as a dime: (A combinations of coins shown) 


91 


90 


9. 


19c purchase, change from a quarter (as above) 


64 


53 


10. 


Monday is second day of week, which day is Thursday? 


46 


38 


11. 


Which clock shows 15 minutes after 7? 




73 


78 


12. 


Which clock shows ten thirty-five? 




61 


56 


13. 


Find the numeral that is one-half of 6. 




85 


81 


U. 


Which box shovrs three- fourths of rectangle dark? 


94 


92 


15. 


Find the numeral that shows one-half of A. 


87 


74 


16. 


True number sentence: 5+7=11 8+6-lA 

6+5-9 3+9-13 




87 


87 


1 / . 


True number sentence: 8«=3+A 10"3+7 




81 


74 




9-5+6 12-6+5 




76 


80 


18. 


Mike scored 4, his brother 2. How to find total: 




6+2 4-2 4+2 2+2 








19. 


Frank caught 6 fish, then no fish. Which picture shows 


87 


85 
82 
84 




how many he caught altogether? 




21. 


Six boys, 8 girls, how many altogether? 

Thrnp cravons. 6 cravons. 7 crayons. How many altogether? 


87 
85 


22. 


Twenty-eight children, 5 absent. Wliich 


shows hov; to find 


72 


59 




number at school? 2^i-5, 28-3, 28+5 


. 23-5 


71 


60 


23. 


Which number line shows 45+3? 




24. 


Leroy gets up at seven o'clock. He gets 


to school two hours later. 


62 
53 


63 
59 


25. 


What time does he get to school? 

Where is 8 the missing number? i5-D»7 

13«D-6, 16-D-7 


lA-D-5. 






Number of Students 


259 


282 






Mean Test Score 


18.99 


18.23 






Standard Deviation 


5.06 


4.80 






Mean Pretest Score 


50.25 


50.14 






Correlation with Pretest 


.65 


.55 



CSMP students did relatively best (at least nore than 10% correct) on items 
4, 9, 15, 22 and 23; five rather unrelated items. On only six of the 25 items, 
items 3, 6, 7, II, 18 and 25, did Non-CSMP students do better (by at most 7%) 
and these items were also rather unrelated. 
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Comparison Tests 

Four comparison tests were developed, two to be individually-administered 
and two group-administered. Each had from two to five subtests; a total of 12 
subtests in all. The tescs were intended to measure students' abilities to do 
certain tasks thought to be related to CSMP; they did not cover specific CSMP 
content. A wide variety of sources was investigated to determine the avail- 
ability of potentially useful tests. This included published achievement tests 
and tests of mental ability, Kit of Reference Tests for Cognitive Factors 
(Educational Testing Service, 1963), problems from the California Mathematics 
Improvement Program, and various research journals in mathematics education. 
Some of the tests finally constructed were developed by the CSMP Evaluation staff, 
others were adapted from available instruments. 

Each of the group-administered tests was tried out in four second grade 
classes and each of the individually-administered tests was tried with at least 
12 students, never more than three from a given class. None of these classes or 
students was in any way involved in the pilot study. Several promising tests 
had to be discarded for one reason or another. 

The 12 subtests listed below are essentially independent tasks, 
though some ara related in various ways. Thus, although total scores for each 
of the four comparisons are also analyzed in this chapter, they are not 
homogeneous in testing a series of related and similar tasks and it is the 
subtest scores >/hich provide the most easily interpretable information. The 
subtests named below will be described in detail in the next four sections 
(one for each test) of this chapter. 



Group Administered Indivl diially Administered 

la): Word Problems TIa): Equacions-Construction Ilia): Classification IVa) : Nambcr Pattemn 

lb): Number Puzzles lib): Kq\iat ions-?Uuency Illb) : Binary Relations IVb) : Functions 

Ic) : E«tir.ation-C«lculations lie): Cocblnatorics IVc) : Probability 

Id): EBtimation-Largeut Nvxaber Ild) : Regrouping 
le) : Showing Fraction* 



Comparison Test 1 (Group Administered) 

For each of the five subtests, a brief description of the tests is followed 
by an item analysis in which the reader can see the actual test items and the 
percent correct for CSMP and Non-CSNP students. Actually the items have some- 
times been abbreviated, to keep reasonably concise the tables which follow. 
These item analyses were calculated by summing across students: 115 CSMP students 
who had a mean pretest score of 49.0 and 120 Non-CSMP students who had a mean 
pretest score of A9.5. 

After the item analyses have been presented, and the reader is perha]js in a 
better position to nderstand V>'. nature of the subtests, the Analysis of Class 
Means is given and this provides a statistical comparison between the six pairs 
of CSMP versus Non-CSMP clasr.es. 
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Test la) Word Problems: A series of rather difficult items (mean score less 
than 5 out of 14 items) was read to the students, one item at a time. The 
student copy of the tests gave necessary pictorial informrition and repeated in 
short phrasfts the important part of the question. The response was open-ended; 
students produced the answer they thought correct. The questions dealt with 
what are usually called word problems (verbal problems requiring an aritlmietic 
operation) as well as problems related to ideas of negative integers, even-odd, 
division, remainder, average and combining operations. Of course words like 
these were not used. The problem involving negative integers, for example, 
dealt with temperatures above and below zero. 



Table U 



1. 
2. 

3. 
A. 
5. 
S. 

3. 

9. 
10. 
!!• 

12. 

13. 

K. 



Item Analysis 
Comparison Test la)'. Word Problems 



Test IcemA 



Small 



Pcrc«nt Correct 



CSMP Non-CSMP 



Medium 



Large 



3c 5C 8c 

Bill Vad 13c and bought 1 large. How much change? 
How much do 3 smalls and 1 medium cost? 

Mary haa 20C and bought 2 smalls and 1 medium. How much change? 
One large costs 1/2 as much as a sucker. A sucker costs? 
One medium costs AC lesr, than a banana. A banana costs? 
Four glasses fill one quart. How many quarts could be filled 
with 13 glasses? 

(From above) How many glasses left? 

A car circles a racotractc 3 times in 10 minutes. Rnw p^ny 
tf.mes in 30 minutes? 

each of the four snakes (picture of A snakes of different 
lengths) grows 1 inch per year. Which doubles his length first? 
The temperature was 4 decrees below zero. Then it got 10 degrees 
warmer. What was the temperature tlicn? 

\ boy is first in line, then a girl, then a boy, .nd so on. 

is the 37th person in line a boy or a girl? 

(vrom above) Is the 54th person a boy or a girl? 

A rabbit weighed 4 ounces, than gained 2 ounces, then doubled 

its weight. How much did it veigh then? 

Tora's t>core was 10, Bill's was u a:,d John's was exactly half way 
between them. What was John's score? 



Mean Sco.ro 
Standard Deviation 
Correlation with Pr«ceac 
KIL20 



70 
52 
26 
11 
23 
41 

44 
17 

7 

17 

49 

46 
26 

29 



4. 58 
3.06 
.60 
.77 



65 
46 
23 
5 
12 
36 

46 
19 

8 

6 

54 

52 
22 

20 



4.12 
2.62 
.52 
.70 
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The reader is reminded that these statistics are derived across students. Mean 
scores will differ by small amounts from those derived across classes in testing the 
significance of CSMP - Non-CSMP differences (p. 21). KR20 (bottom line) is a measure 
of the homogeneity of this set of test items; the degree to which they test the same 
underlying ability. It usually varies from 0 (zero average correlation between items) 
and 1 (high correlations). 

CSMP students did slightly butter on items A and 5 dealing with what might be 
tailed i:onvt?rst' rt-lat ionsli Ips , on item 10 which dealt with negative integers and on 
item 14, implicit averaging. These items, while they are applications of the CSMP 
content, are quite different from anything in the CSMP curriculum. Non-CSMl^ studentis 
did slightly better on items 11 and 12 regarding odd and even, though all percentages 
were generally close to that expected from guessing alone. 
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Test lb) Number Puzzles: Given a completed addition or subtraction problem 
but with one or two digits from the original question missing, students were to 
figure out the missing digits. For example 36 

19 

Table 12 



Item Analysis 
Comparison Test lb): Number Puzzles 







Percent Correct 




lest item 


CSMP 


Non-CSMP 




5 








8 






1 

i • 




62 


66 




i / 






2. 


29 




"7 T 
/ J 


J. 


-H/d 


"7 1 
/ J 


7 3 




58 






4. 


08 


9 


5 


5. 




22 


11 




51 








36 






6. 


-l9 




4 




19 






7. 


5,9 


6 


10 


8. 


-07 


6 


7 




24 








Mean Score 


2.62 


2.50 




Standard Deviation 


1.33 


1.22 




Correlation with Pretest 


.45 


.50 




KR20 


.41 


.40 



Items A-8, involving addition with carrying or subtraction with borrowing, 
were very difficult for students. The Jar-^est .fference was on items A and 5 
involving carrying. 
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Test Ic) Estimation, Calculations: Oi a very large sketch pad at the front 
of the room a calculation was shown to the class for about 10 seconds, including 
time for the tester to read the calculation. Students then chose the one of 
three alternatives shown in their test booklet which they thought closest to the 
answer to the calculation. For example with a calculation of ^x78 and alternatives 
of 30, 35, AO, one might think '*78 is close to 80 and %x80 is AO'* or "2x35 is 70 
and 2x40 is 80, so AO Is closer." There was not sufficient time for students to 
sctually calculate "^x78»39 and AC is closer to 39 than is 30 or 35." Two practice 
items were eiven before the actual teat items. 

Table 13 



Item Analysis 
Comparison Test Ic) ; Estimation - Ca^ ■:ulations 











Percent Correct 


Calculation Posed 


Alternatives 


CSMP 


Non-CSMP 


1 . 


19+17 


35, 40, 45 


70 


62 


2 


95- 


-34 


50, 60, 70 


48 


C ; 1 


3. 


1/ 


^x78 


30, 35, 40 


50 




A. 


2x63 


120, 125, 135 


4J 


38 


5. 


33+46 


70, 80, 90 


34 


37 


6. 


96 


-78 


20, 30, 40 


26 


35 


7. 


325+582 


800, 850, 900 


49 


54 


8. 


1/2x73 


30, 35, 40 


45 


39 


9. 


22+63 


80, 85, 90 


59 


54 


10. 


4x19 


40, 60, 80 


35 


45 


11. 


399-201 


100, 150, 200 


32 


33 


12. 


1/3x63 


20, 25, 30 


23 


13 






Mean Score 


5.10 


5.08 






Standard Deviation 


2.23 


1 .88 






Correlation with Pretest 


.41 


.26 






KR20 




.49 


.25 



This test was certainly a speeded test and was intended to be. Students 
were to "guess" about what the answer would be. Thus there was undoubtedly 
considerably real guessing taking place (a stab in the dark) and this is 
corroborated by the moderate correlation with pretest and the moderate KR20, Though 
ttiere are differences between the two groups of student.3 on in(3 Ividual items, the 
differences are not systematic according to either arithmetic operation or 
size of number. 



Te«t Id) Estimation: Largest Numbers. As before, students were shown briefly 

^hL''^i^n °T °" ^u^""" ^ '^^"^^ °" ^ish^ in ^ square. 

They then had to mark on their own page the (empty) circle or square, depending 

on which calculation shown at the front of the room that they thought produced 
the largest answer. 



Table U 



Item yVnalysia 
Comparison Test Id); Lstimation - Largest Number 





Percent Correct 


(Students to 


select larger) 


CSMP 


Non-CSMP 


i. 


6329 


, 6328 


50 


54 


2. 


527+84 


, 52 7+86 


90 


79 


3. 


ix250 

2 


, ix250 
3 


63 


21 


A. 


19x3 


, 18x3 


86 


87 


5. 


27+-80 


, 26+82 


62 


59 


6. 


300-24 


, 300-23 


41 


30 


7. 


19+7+5 


, 5+19+8 


83 


79 


8. 


700-20 


, 710-50 


45 


39 


Mean Score 
Standard Deviatio.i 
Correlation with Pretest 
1CR20 


5.21 
1.52 
.48 
.38 


4.48 
1.46 
.32 
.31 



This subtest appears similar in some ways to the previous one. Actually 
it does not require computational facility, but rather what might be 
intuitive ideas regarding ccmniutativity and order properties. CSMP students 
did better on several items, noiably items 2 and 6 and especially item 3 
dealing with multiplication by 1/2 or i/3. Note that on pure calculations 
with 1/2 and 1/3 (items 3, 8 and 12 of subtest Ic)) there was very little 
difference between the two groups. This test also was characterized by moderate 
pretest correlation and low KR20, perhaps due in part to guessing. 
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Test le) Showing For each of four shapes, usiia3.1y partial grids, students 
were asked to shade a given fraction of the shape. ?nr example, shade 1/3 of 



Table 15 



Item Analysis 
Comparison Test le) : Showing Fractions 



Test Items 

(Student to shade indicated fraction of given figure) 



Percent Covrect 



CSMP 



Non-CSMP 



1. 1/3 



10 



2. 2/3 




40 



3. 1/6 



4. 1/2 



21 



43 



14 



19 



Mean Score 
Standard Deviation 
Correlation with Pretest 
KR20 



1.13 
1 .00 
.45 
.35 



0.9 
.76 
.24 
.02 



Item 4 required the students either to shade every other square or to count the 
total number of squares and shade half that many. Tliis was the only item from 
the five subtests of Test I that was similar to an exercise appearing in the CSMP 
curriculum and there is a fairly large difference in performance for that item. 
Once again the correlations V7ith pretest are moderate and the KJ120, particularly 
for Hon-CSMP students, was low. 
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Analysis of Class Means: Table 16 presents the important summary data in 
the comparison of CSMP and Non-CSMP classes for Test I, It gives for each group 
the mean score across the six CSMP and the six matched Non-CSMP classes for 
each of the subtests, ror the total, and for the pretest. These means are 
slightly different from those in the item analyses because they were devised 
across classes rather than across students. A-so given in Table 16 are the 
probability levels at which the observed differences are significant. 

Table 16 



Analysis cf Covariance 
Comparison Test I 





Mmd Test Score 
Across Classes 


Mean Pretest Score 
Across Classes 


F-test, 1 and 4 
degrees of freedom 
p less than: 




CSMP 


N-CSMP 


CSMP 


N-CSMP 


■ la) Word Problems 


4.79 


A. 06 








.23 


lb) Number Puzzles 


2.66 


2.51 








.56 


I^) Estimation: Calculation 


5.18 


5.08 




49,49 


49,01 


.95 


lo) Estimation: Largest Number 


5.26 


4.44 








.04 


le) Shoving Fractions 


1.15 


0.92 








.18 


Totals Test X 


19.06 


17.01 








.12 



All differences were in favor of CSMP, though significance was reached at 
the ,05 level on only subtest le) Estimation: Largest Number. The adjustments 
in mean scores due to the differences in entering ability (not shown) were 
small, the largest such admustment being .10 on the total score. 
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Comparison Test II (Group Administered) 

The format of presentation is the same - first the test descriptions and 
item analysis for each of the subtests, then the summary comparison of class 
means. The former are based on 145 CSMP students with a mean pretest score 
of 53.14 and 146 Non-CSMP students witli i mean pretest score of 52.77. Seven 
pairs of classes cook this test. 

Test Ila) : Equations - Construction. Students were given a string of 
symbols (numerals, operation signs and equality sign) and asked to form an 
equation (number sentence) with them. Each symbol had to be used exactly 
once. For example, with the stiing =,-!-, 2, 5, 3 one ^lould make 3+2>=3 
(or 2+3=5 or 5=2-^3 or 5^-3+2). Thus these v;era essentially problems with closure; 
to find the equations whicli worked. 



Table 17 



Item Analysis 
Test Ila) : Equations-Construction 



Test Items 


(Given strings of symbols) 


Percent Correct 


rrom 


wnicn equation was 


to De constructea; 














CSMP 


Non-CSMP 


1. 


+ 


8 12 




96 


96 


2. 


X 


6 18 


3 


90 


91 


3. 


= X 


15 5 


1 


71 


03 








3 








+ 


5 


21 6 10 




10 


5. 




X 2 


32 10 3 


3 


1 






Mean Score 


2.73 


2.02 






Standard 


Deviation 


.88 


.58 






Correlation with Pretest 


.53 


.31 






KR20 




.46 


.28 



Only on item 3 was there nuch difference. On that item, which dealt with 
multiplying by \ more than two-thirds of the CSMP students were correct while 
virtually none of the Non-CSMP students were successful 



2^ 

ERIC 



23 



Test lib): Equations - Fluency. This test was similar to Test Ila) in that 
students were to produce equations from strings of symbols. This time, however, 
the objective was to produce as many equations as possible using any or all of 
the given symbols as many times as desired. The numbers were small, the emphasis 
on putting combinations together appropriately ratlier than on computational speed. 
The score for each string of symbols was the number of correct equations in the 
minute time allowance. 

Table 18 



Item Analysis 
Test lib) Equations-Fluency 



Given String of Symbols 


M©«n Number of Correct Etjuatlons 


CSMP 


Non-CSMP 


1. - -i- - 2 4 6 

2. - + - X 1 2 3 

3. - + X i- 1 2 4 

2 


3.27 
4.90 
5.42 


3.55 
4.34 
3.42 




Total 


13.57 


11.31 



CSMP students did better on the second item and much better on tlie third 
item, where "i" and "x" were given. A separate count was made in the third 
item of correct equations containing *'x*'. For CSMP students the mean number 
was 3.33 and for Non-CSMP students it was 1.61. The mean number of correct 
equations using was 1.48 for CSMP students and 0.10 for Non-CSMP students. 
Also on the third item, the mean number of incorrect equations produced was 
1.12 for CSMP students and 2.14 for Non-CSMP students. 

Test lie): Combinatorics. Both situations given to the students asked 
to list the ways of doing something. 



Table 19 



Item Analysis 
Test lie) Combinatorics 



2. 



Situations Foaed 



Given packages of gum, aa pictured below (the 
number on the package tells how many pieces 
are in the package), show ways of getting 
exactly ssven pieces. Students were actually 
given multiple copies of picture below and 
put X's on appropriate packages. 
QOQQ CD CD 

List as nany tvo digit nunbers aa you can 
from tha difiita 3 and 7. 

Total 



Haan Nunber of Ways Shows 



CSMP Non-CSMP 



5.01 



11.97 



16.98 



A. 99 



12.17 



17.16 



CSMP students might have been expected to do better on these items since they 
have occasional opportunities to deal with combinatoric situations. However, there 
was virtually no difference in scores. 
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Test Ild) : Regrouping. Two kinds of situations were given involving the 
filling with water of three sizes of cans: large, medium (2 mediums - 1 large), 
and small (2 smalls = 1 medium). In one situation, a certain number of 
particular-. sized cans was available to fill as many of a particular larger 
sized can as possible, i.e., how many could be filled, and how man)^(^.^of the 
smaller cans would be left over. In the other situation given numbers and 
combinations of medium and large cans were shown and the student was to 
determine how many small cans could be filled from them. These situations 
also required extensive preparation by way of examples in preparing 
students for the situations and ways of responding. The items 
presented pictorially, but are given verbally in Table 20. 



Table 20 



Item Analysis 
Test lid) : Regrouping 











Test Items 






Percent Correct 
































CSMP 


Non-CSMP 


1. 


Given 


4 smalls. 


How many 


mediums can be filled? 






78 


79 


2. 






How many 


smalls left over? 






75 


71 


3. 


Given 


7 mediums 


. How many 


large can be filled? 






63 


71 


4. 






How many 


mediums left over? 






71 


73 


5. 


Given 


10 smalls 


. How many 


large caai be filled? 






34 


42 


6. 






How many 


smalls left over? 






38 


41 


7. 


Given 


15 smalls 


. How many 


large can be filled? 






23 


28 


8. 






How many 


smalls left over? 






20 


26 


.9. 


Given 


17 smalls 


. How many 


mediums can be filled? 






18 


27 


10. 






How many 


smalls left over? 






35 


41 


11. 


Given 


3 mediums 


. How many 


smalls can be filled? 






77 


78 


12. 


Given 


1 large. 


How many 


smalls can be filled? 






54 


: 52 


13. 


Given 


1 large. 


2 mediums. 


How many smalls can be 


filled? 


45 


45 


14. 


Given 


2 large. 


3 mediums. 


How many smalls can be 


filled? 


32 


29 


15. 


Given 


5 large. 


4 mediums. 


How many smalls can be 


filled? 


12 


17 




Total 


6.76 


7.20 



This subtest dealt with an instance of regrouping but was perhaps only 
tenuously related to concepts of place value (regrouping in tens) Non-CSMP 
students did better on the first ten items (small units combined into larger 
ones) while there was no difference on the last five items (larger units broken 
down into smaller ones). Though the differences are not significant in either 
case, one might speculate^thdt while CSMP uses primarily one pedagogical 
technique for place value (the Minicomputer), Non-CSMP classics may use a 
variety of techniques (bundles of sticks, blocks, etc.) which might or might 
not make them as proficient with place value in tens but which might help 
them more in other regrouping concepts. One of the Tests of Standard Content 
administered to these students in first grade dealt with filling bags with 
blocks, ten at a time. CSMP students did better on that task. 
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Analysis of Class Means: 
classes. 



The data in Table 21 are based on seven pairs of 



Table 21 



Analysis of Covariance 
Comparison Test 11 



Test 


Mean Test Score 
Across Classes 


Mean Pretest Score 
Across Classes 


F-test, 1 and 5 
degrees of freedom 
p less than 


CSMP 


N-CSMP 


CSMP 


N-CSMP 


Ila) 


Equat ions-Cons tructicn 


2.72 


2.03 






.01 


lib) 


Equations-Fluency 


13.63 


11.19 






.02 


lie) 


Combinatorics 


16.94 


17.08 


52.46 


52.15 


.73 


Ild) 


Regrouping 


6.34 


6.55 






.40 




Total 


39.62 


36.86 






.04 



Significant differences were recorded on Subtests 11a) and lib) dealing with 
the construction of equations and on the total of Test II. Again the adjustments 
in mean scores due to differences in pretest scores were small, the largest being 
.U for the total score (i.e. CSMP adjusted mean = 39.48, Non-CSMP - 37-00)- 
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Comparison Test III (Individually Adndnistered) 

The item statistics for the two subtests of Comparison Test III are based 
on 81 CSMP students with a mean pretest score of 53.4 and 78 Non~CSMP students 
with a mean pretest score of 54.1. 

Test Ilia) Classification. There were two parts to this subtest. On the 
first part the student worked with a 4x4 array, shown below in reduced size. 
The figures were actually colored red or blue as indicated and were identical 
except for differences on the four attributes: color (red, blue) > height (tall, 
short), width (fat, thin) and sex (boy, girl). On the second part students were 
given a set of 27 cards in three colors (red, blue, yellow), three shapes (square, 
triangle, circle) and with varying numbers of dots marked on them (one, two or 
three). That is they were given cards with each possible combination of 
attributes except that one card was mi^.sing and they were to figure out which 
card was missing. As before the items in the item analyses are abbreviated and 
do not reflect the careful explanations, repetitions and encouragement ^^iven by 
the tester. 




Figure 1. Picture of Children used in Test Ilia) 
R: Colored Re<i, B: Colored Blue 

ERIC ^^'^ 
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Table 22 



Item Analysis 
Comparison Test Ilia): Classification 
Items 1-10: Figures, Items 11-13: Shapes 



Vu chinking of ona oi :htS0 chlldien but L'o 
woz going to tall you which one. Find out aa 
m»ch AS you can about the one I'n thinking 
•bout, but you un only ask three questions* 
Th»y have to be questions I car. answer with YES 
or NO. ("Target" was taJl, thlr., red girl.) 



i Attending to: 
I He'.ghc 
{ Width 
1 Cnlot 
Sex 
Total 
Non-Dlf f e rent tab 1 e ^ 



Percfnt Correct 



43 

74 
67 
72 
2.36 

12 



Non-CSMP 



38 
86 
60 
67 
2.M« 
18 



2. Mom which ona do you think it is? (the criterion 
wa whether choice was consistent with responaea 
U 1.) 

3t Uh«t can you tell me about thia one (tall, 
tbin, blue boy)? Uow is it different {ron 
thtt othara2 



Height 

Width 
Color 
Sex 
Totsl 

I Mon-Differentlable^ 



31 



36 
72 
67 
62 
2.57 
23 



28' 



58 
65 
49 
68 
2.40* 
45 



4. ULai 



t can you teLl ae about this one (point to 
a^rt, fat, red girl)? How is it different 
froa the others? 



Height 
Width 
Color 
Sex 
Total 
Von-Differentlabla^ 



52 
89 
54 
70 
2.65 
33 



50 
02 
49 
64 
2.45" 
53 



5. rind the short, thin, blue girl. 



91 



92* 



8. foiat to ill tJ.e ones 'jhlch are thia acd red. 



j Two or nore correct 
All four correct 



99 
65 



99« 
69< 



7. PoJjit to all the ones which are short and 
mot fat. 



Two or Bore correct 
All four correct 



89 
80 



90a 
79« 



t. (After rcBovlng picture) How many red boya 
ware there In the picture? 



Rasponao of 2,4 or 
Raaponae of 4 



52 

.'7 



46* 
27« 



9. (After putting picture back and covering 
tha ehorc, fat, blue girl) What can you 
tall aa ab«ut the one that'e missing? 



10. Look oc thie one (the short, fat, red girl). 
Which one ia most dlfferCtiw 'rom it? Ln 
what waya ia It different? 



Height 
Width 
Color 
Sex 

All four correct 



49 
91 
80 
94 
16 



Height 
Width 
Color 
Sex 
Total 



43 
85 
48 
72 
2.48 



53 
94 
64 
90 
17« 



37 
81 
41 
57 
2.26* 



11. (After ahowlng etudent unsorted sec of 26 
shape carde.) Now ona of these pieces Is 
■isaing. Do you reansber with the picture 
of the children you had to figure out which 
ona wee mlaaing? Well this is the saod thing. 

*K4an Scor;^ (Sim of itons with aeterlsk) 
Standard Oevietioo 
Correlation with Pretaat 
020 



Identified: 
Color 
Shape 

NvBsber of Dots 
Totel 



68 
78 
65 
2. LI 

18.01 
4.66 
.53 
.74 



79 
78 
46 
2.03< 

17.22 
3,47 
.52 
.51 



^aoo*dlffe restlabte ettrlbutaa refer to hair, ayee, mrmM «nd other charaetariatice 
irtilcli era la fact identical for tha plceurad childran. Care had to be axarciaad when 
ftha charaetarlatic, euch ac hair, was uaad to dtf f araatiata boya mad girla, la which 
caaa it waa a lagitlMata reeponae. Raspooaaa dealing with nwa or coluno ware givea 
•cesalooally and were accaptad. 



As can be seen, CSMP students did slightly better on almost all the items though 
the differences were small in all cases. Whatever vocabulary the child used was 
accepted and continued after it was determined what was intended by the word (long, 
wide, round, square and many other words were used). On item 11, if the student did 
not get anything he was asked specifically what color the missing piece was, and then, 
if necessary, told the color and asked the shape. Thus each of the three items were 
in some sense independent. In assigning a total score partial credit was given in 
items 6, 7 and 8, and for item 9 credit was given only if the student gave all four 
characteristics of the missing piece. 
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Test Illb) Binary Relations. For this subtest, a three dimensional represen- 
tation like that below was shown to the students. A turtle sits on the railway 
track looking down the tracks in one direction or the other (i.e., towards either 
the mountain or the river). On his back sits a bug who also looks in one direction 
or another, though not necessarily the same as the turtle. 




Figure 1. Demonstration Material for Test Illb) (top view) 



At the sound of a Train whistle (T for. turtle) the turtle turns around and 
faces the opposite direction. The bug takes no action, but of course he too 
ends up facing the opposite direction. At the sound of a Bell (B for bug) the 
bug only turns around, The question then was to determine, for a given starting 
position, what the final positions of the bug and turtle would be after a 
particular series of train whistles and/or bells. With a little thought it can 
be seen that the turtle faces the same or opposite to his starting position 
according to whether the number oi" train whistles is even or odd respectively, 
and the bug faces the same or opposite according to whether the nximber of sounds 
(train whistles and bells) is even or odd and that the order of these sounds is 
jjnmaterial. In the item analysis below> initial starting positions are given 
by "right" and "left", though on the actual test the realistic looking bug and 
turtle were actually placed to look at the mountain (left) or river (right). 
Students were not allowed to pick up the animals. 

The materials were taken away for the last two questions, which were orally 
given problems involving a hypothetical flea crawling up a hypothetical flag pole. 
These items (see Table 23) touched on the idea of composition of functions. 
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Table 23 



Item Analysis 
Comparison Test Illb): Binary Relations 



Tesc Icens 



Iniclal Posicion 

Bug - right 
Turtle - left 



Sounds 
T. B 



Final Position 
Bug ? ? 
Turtle ? 



Percent Correct 



CSMP Non-CSMP 



89 
99 



95 
97 



K«g - right 
Turtle - left 



B • T 



Bug ? 
Turtle ? 



Bug - right 
Turtle - laft 



T • T • T -B 



Bug ? 

Turtle ? 



A6 
67 



A7 
62 



47 
67 



53 
72 



Bug - right 
Turtle - left 



T . T • B • T . T • B 



Bug ? 
Turtle ? 



63 

5S 



35 



Bug - right 
Turtle - left 



10 train whistles 
then 5 bells 



Bug ? 
Turtle ? 



59 
62 



67 
6A 



Bug - right 
Turtle - left 



What 3 sounds could 
produce ^ 

- right 
left 



:er 
f Bug 
/Turtle 



Bug ? 
Turtle ? 



(After taking away material) Suppose 
a flea is climbing up a tall flagpole. 
When he hears a bell he cliinbs up 3 
Inches, then on the next bell he slips 
back 2 inches, then up 3 inches, then 
back and so on. 



How far up the flagpole will he be after 
1 balls? 



32 



33 



41 



26 



How far up will he be after 
100 bells? 



Response < 100 
Response between AO and 60 



77 
9 



79 
lA 



M«an Score 
Standard D«vl ^ on 
Correlation vitn Pretest 
KR20 



8.01 
1.95 
.17 
.30 



8.00 
1.97 
-.02 
.3A 



The statistics for the two groups of students were almost identical. For 
some reaFon CSMP students did better on item A, but Non-CSMP students made up 
for that over the rest of the items. The test was difficult, the responses to 
items 2 to 5 being near the guessing level, and the correlations and KR20 being 
very low. 
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Analysis of Class Means: Xlze data in Table 24 are based on 12 pairs of 
clasres, with from five to seven students in each class (selection of students 
is d.jscribed on p. 7) taking the tests. 



Table 24 



Analysis of CoVariance 
Comparison Test III 





Mean Test Score 
Across Classes 


Mean Pretest Score 
Across C}.a6ses 


F-test, 1 and 10 
degree of freedom 
p less than 


CSMP 


Non-CSMP 


CSMP 


Non-CSMP 


II In) Classification 
Illb) Binary Relations 
Total 


17. 9& 
8.05 
26.03 


17.23 
8.12 
25.36 


53.00 


54,29 


,71 
.37 
• 28 



None of the differences were even close to significance. Using class means, 
the correlation between test and pretest range from -.23 to -.50 for the three 
scores, so that though CSMP classes had the lower pretest score, their mean test 
scores were actually adjusted downward while the mean scores for Non-CSMP classes 
were adjusted upwards. The adjusted mean total scores, for example, were 25.80 and 
25.59 for CSMP and Non-CSMP classes respectively. The explanation for the 
differences between the positive con;elations , when computed on an individual 
student basis, and the negative correlations, when computed on a class mean 
basis under the analysis of covariance model, is that within schools, i.e., 
class pairs, the class with the lower pretest score more often than not had a 
higher test score. Thus interpretation of this test is difficult. Perhaps the 
simplest summary statement is x^?hat has already been said - that there were no 
significant differences. 
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Comparison Test IV (Individually Administered) 

The item statistics for these tests were based on 78 CSMP students with a 
mean pretest score of 53. lA and 74 Non-CSMP students with a mean pretest score 
of 54.35 

Test IVa) Number Patterns: Students were shown a card, with a series of numbers 
and blank somewhere, such as 5, j6, 2» 1» and asked what could go In the blank. 
They were also asked to give the reason for their selection. If an unexpected 
answer was given they were oskeato explain that answer and if they couldn't, to 
find out if any other .:umber could go there. It was possible, though unusual, for 
a student to give a correct reason but to make an ai'thmetic error. Thus two parts 
were scored for each item. 

Table 25 



Item Analysis 
Comparison Test IVa): Number Patterns 



Test Item 


Percent Correct* 


CSMP 


Non-CSMP 


1. 


5, 


6. 




100 
95 


100 
99 


2. 


2, 


^. 




91 
92 


92 
89 


3. 


19, 


1 


7. 15. _. 11 


74 
68 


66 
66 


4. 


15, 


2 


0, 25, 30, 


90 
79 


86 
69 


5. 


" 1. 




^ 8, 


17 
17 


3 
5 


6. 


16, 


n, _, 7, A 


62 
58 


55 
54 


7. 


0, 


5, 


1, 6, 2, _ 


29 
22 


28 
23 


8. 


5. 




13, _, 21 


63 
55 


55 
50 




Mean Score 
Standard Deviation 
Correlation with Precast 
ltR20 


IC.ll 
3.52 
.54 
.85 


9,.42 
3.53 
.62 
.86 



*The second entry for each item is the percent who gave 
a correct reason for their choice of number. 
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CSMP students did better on almost all items, the differences usually being 
rather small. The largest differences occurred on item A, which was essentially 
counting by fives. l^Hiile these items probably test a rather general nvimerical 
ability, items similar to them often being found on tests of mental ability, CSMP 
students do considerable work with arrow diagrams where number patterns emerge in 
striking ways. They do not, liowever, do items such as these and therefore these 
results are encouraging. 

Test IVb) Functions* For this test, the student was told that a teacher 
was playing a game with a class. Every time' a student gave her a number the 
teacher did sorrething, always the same thing, to the number and got a new 
number. The student was then shown a series of number-pairs on a card like the . 
following: 



A 


changes 


to 


7 


11 


changes 


to 


14 


5 


changes 


to 


8 



This means that after the mysterious "something" was done to the ninnbers, 4 was 
changed to 7, 11 14 nnd 5 to 8. \^at was it that was done to each number? 
If a student gave no explanation or an incorrect one, and only then, he was 
asked "If we did the same thing to 7 what cculd we get?" Thus the student could 
get the item correct by explanation or^ failing that, by example. If the 
student gave a correct but incomplete explanation, such as "made it larger" for 
the example above, he was encouraged to be more precise ("Yes that is correct, 
but they are larger in a special way") . 

For each series of number-pairs a second question was asked. This time the 
number the teacher ended up with was given and the question was what number must 
the student have given her. In the example above, the number that the teacher 
ended up with A (not shown) and therefore the number she started with must have 
been 1. 



3V) 
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Table 26 



Item Analysis 
Comparison Test IVb) : Functions 











Percent Correct 
By Explanation 


Percent Correct 
By Example 


Percent Who Got 
Original Input 






CSMP 


Mon-CSMP 


CSMP 


Non-CSMP 


CSMP 


Non-CSMP 


1. 


1-2. 


6-7, 


4-5, (16) 


90 




97 


3 


3 


88 


86 


2. 




11-14, 


5-8, (4) 


86 




82 


3 


7 


67 


57 


3. 


3-6, 


5-10, 


7-14, (12) 


37 




22 


28 


16 


21 


18 


4. 


27-25, 


A-2, 


15-13, (17) 


78 




74 


0 


3 


33 


31 


5. 


10-5, 


2-1, 


6-3, (6) 


21 




24 


12 


16 


13 


15 


6. 


2-6, 


4-12, 


1-3, (9) 


13 




3 


3 


1 


10 


1 


7. 


>7, 


2-5, 


5-11, (13) 


3 




3 


4 


1 


3 


1 


8. 


Johnny gave the teacher a number, 
added one to it and ended up with 
nufflber? 


She doubled it and then 
9. What was Johnny's 


32 


23 




Mean Score 
Standard Deviation 
Correlation with Pretest 
KR20 


6.45 
3.31 
.50 
.81 


3.86 
3.19 
.55 
.82 



*Tha nuabcx in parentheaea la the number given In the aecond part of the 
Iten, i*a*, '%ftiat nunber did the student give the teacher for her to end up 
with thia number?" 



Again the differences are small but consistently in favor of CSMP students. 
This may have been a result of the considerable woik done in the CSMP curriculum 
on return arrows. 
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Test IVc) Probability. In this test the student was consider that painted 
blocks were rolled, like dice, to determine the color that would most likely appear 
on the top face after the roll. Several blocks were used, with different numbers 
of sides painted various colors. A block with two sides red, three sides blue 
and one side yellow is denoted by 2R-3B-1Y. Actual blocks with 1-inch sides were 
used and students were to pick them up and examine them but not to roll them. 



Table 27 



Item Analysis 
Comparison Test IVc): Probability 



Test Item 


Fercttnt Correct 


CSMP 


Non-CSMP 


1. 


Given 5R-1B. What color is most likely to end up 
on top? 


89 


81 


2. 


Why did you pick ? 




61 


70 


3. 


If you had some red and blue p«lnt» how could you 
fix that block so that both red and blue have the 
same chance of ending up on top? 


59 


38 


A. 


Given 2R-3B-1Y, 3R-2B-1Y, 4R-2B 

Which one is most likely to end up with blue on top? 


72 


64 


5. 


Given 3R-2B-1Y. Suppose we tossed this block 12 
times. About how many of chose 12 times do you 
chink red would end up on r.op?* 


54 


34 


6. 


Suppose we tossed ±f 12 
many times do you tnink 


times again. About how 
blue would end up on top?** 


55 


61 


7. 


(Response for Item 6 > response for item 5.) 


50 


51 




Mean Score 
Standard Deviation 
Correlation with Pretest 
KR20 


4.60 
1.50 
.37 
.40 


4.08 
1.62 
.19 
.47 



AResponae of 5,6 or 7 counted correct. 
**£ABpoiise of 1»2,2,4,5 counted as correct. 



Again the differences were in favor of CSMP, particularly on items 3 and 5. 
CSMP students had some limited experience in probabilistic ideas, but they had 
not worked with blocks or dice, nor had they been given the notion of a probability 
as a number related to nuniber of ways an event can occur. 
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Analysis of Class Means: The data in Table 28 are based on 11 pairs of CSMP - 
Non-CSMP classes. Usually about six or seven specially selected students from 
each class were administered these tests. 



Table 28 



Analysis of Covariance 
Comparison Test IV 



Test 


Mean Test Score 
Across Classes 


Mean Pretest Score 
Across Classes 


F-test, 1 and 9 
degrees of free<^i'm 
p less than 


CSMP 


Non-CSMP 


CSMP 


Non-CSMP 


Test 


IVa) Number Patterns 


10.11 


9.28 






.05 


Test 


IVb) F\inctions 


6. AS 


5.75 


5^4.18 


5A.52 


.10 


Test 


IVc) Probability 


4.59 


3.99 






-05 




Total Test IV 


21.18 


19.01 






.02 



It can be seen that all differences were in favor of CSMP and all were 
significant except Test IVb): Functions, which nevertheless approached significance. 
The adjustment of mean scores -^ue to pretest differences was small (<.04) in all 
cases. The phenomenon of positive correlations between prer.?.st and test sr.jres 
when based on studen^-s and negative correlations when based on classes recurred 
for Test IVc), the correlation by classes being -.54. 
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CSMP Tescs 



Construction of Tests 

The second grade CSMP curriculum consists largely of a detailed set of 
lesson plans, complemented by a series of workbooks and worksheets. There are 
287 lessons which use a spiral development where a child is introduced to each 
of several topics, a little at a time and then proceeds through increasing 
levels of sophistication with each topic. In each of the eight workbook series 
which complement the lessons, there are seven workbooks graduated in difficulty 
from the first which is almost remedial in nature to the seventh which is 
difficult and intended only for the best student. 

Generally only those topics which had bten covered extensively and with 
which the children had had considerable practice were considered for inclusion 
in the end-ol*-year test. 

The basic source of items was the workbook series as it was felt the problems 
in the workbooks provide the most accurate indication of what was expected of the 
students. Since many of the teachers did not complete all eight of the workbook 
series, only the topics covered in the first five workbook series (about 220 
lessons) were considered. Furthermore all of the content selected from' the work- 
books was from the first five levels of each workbook series. 

The first 220 lessons were analyzed to ensure the content selected from the 
workbooks reflected and included the content of the lessons. A list of topics 
included in the lessons is given below and the number of lessons related to that 
topic is given in parenthesis. About half of the lessons were concerned with 
more than one topic, hence the total number of lessons shown greatly exceeds 287. 



Table 29 



Topics In CSMP 
Second Grade Curriculum 



Topic 



Number of LesBons 



Addition 

Subtraction 

Multiplication 

Integers 

Rationals 

Minicomputer 

Relations and Arrow Diagrams 
Open Sentences 

Classification-Sets and String Diagrams 

Parenthesis 

Combinatorics 

Probability 

Numerals and Counting 

Geometry and Measurement 

Mental Arithmetic 

Order 



93 
61 
53 
30 
31 
50 
59 
18 
20 
18 
19 
7 
12 
17 
18 
10 
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Some of the above topics were covered at least partially on the Standardized 
Test; others, while not tested directly on the Comparison Tests, were at least 
related (in some cases rather vaguely) to one or more of those subtests. These 
topics were not repeated; hence the CSMP tests which were finally constructed 
tended to contain only material not found in other curricula (such as the use 
of the Minicomputer and Arrow Diagrams) or not usually fo\ind at the second grade 
level (such as integers or multiplication by larger numbers) . One certainly 
cannot look on the tests in this chapter as an operational definition of the 
CSMP curriculum; they show, in a sense, the residuals of the tests in the previous 
two chapters. 

Four test booklets (A to D) were constructed and administered to eight or 
nine classes according to the testing plan given on page 7. Each test usually 
required about 20 minutes but was essentially untimed as students worked through 
the test at their own speed. The format of both the test booklets and of the 
test items closely resembled that of the workbooks. Students received help 
("What to do" not "how to do it") from tlie test administrators whenever necessary, 
usually on an individual basis. The first drafu of each test was pilot tested in 
three second grade CSMP classes which were not scheduled in the testing plan to 
take the particular test. 

A brief listing of the content tested on the four tests is given below. 

Test A: Open sentences and parentheses (6 items) 

Whole number multiplication (10 items) 

Arrow diagrams (5 items) 
Test B: Fractional parts of whole numbers (10 items) 

Interpreting an arrow diagram (4 items) 
' Composition of functions: arrow diagrams ( 3 items) 

Test C: Integers: ordering and adding (20 items) 

Multiplication - ordering (7 items) 
T£St D: "Solution" of arrow diagrams: see item analysis (5 problems) 
In addition Tests A, B and C each had a four-item problem on the 
converse of a relation (return arrows). 

A fifth test, denoted Test E, was administered to a representative sample 
of six students from each class of the classes shown in the testing plan. This was 
an individually administered test of students' ability to use the Minicomputer • 

Results 

The results will be presented in the form of item analysis. Within each 
test items were grouped together logically to make up subtests. The order in 
which the subtests are ^Iven \\>ce was chosen tor ease of interpretation and 
does not correspond to the order of the tests themselves. The statistics were 
derived by averaging across all students who took the test and this includes 
data from classes 14, 15 and 16, which classes have not been considered in 
previous analyses since there were no appropriate comparison classes available 
for them. In the previous two chapters, all students who were in the class in 
the fall and present for the pretest were considered for both item analyses and 
class means. In this rhaptei only students who were in CliMV first grade last 
year (for at least the last 'our months) and who were pr -serxt for the Pretest 
are conL-I.dered. Diffarencef^ between these students and those who started CSMP 
in aecond grade will be ccnjidered for all tests in the next chapter. 
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The mean pretest scores for students who took tests A, B, C and D respectively 
were 50.7, 52,3, 50.6 and 50.3, and correlations between the pretest and each of 
the tests were also remarkably similar: .70, .64, .68, and .60 respectively. 

Table 30 



Item Analysis 
CSMP Subtest 1 



Test Items (from Test A) 



Percent Correct 



1. 


2 


X 


6 = 


2. 


8 


X 


0 = 


3. 


3 


X 


3 = 


4. 


2 


X 


13 = 


5. 


3 


X 


12 = 


6. 


2 


X 


37 = 



83 
64 

86 



67 
49 
44 



Mean Score 

Correlation with Pretest 
KR20 



3.93 
.64 
.78 



The percent correct for items 1-3, Table 30, is in the same range as for 
similar items on the Standardized Test. The percents correct for items 4-6, not 
covered on standardized tests at this grade level, are lower, but still quite 
high for items of this nature. Particularly jjnpressivc, to the author at least, 
is th.e fact that 44% of the students could calculate "2x37" correctly. 
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Table 31 



Item Analysis 
CSMP Subtest 2 



Test Items (from Test A) 


Percent Correct 


1. 2 X 37 » 


40 






2. 2 X 45 = 


38 


3. 2 X 13 = 


60 


4. 3 X 14 = 


38 


Mean Score 

Correlation with Pretest 
KR20 


1.76 
.55 
.86 



The items in Table 31 are similar to those in Table 30. The difference Is 
•J^M in Table 30 appeared first in the test; later came the items in 

Table 31. These latter items included a clue for the students; a reminder of a 
way in which they had been taught to do these problems. Items 4 and 6 of Table 
JO are in fact the same as items 3 and 1 res.^ectively of Table 31. Contrary to 
!^rr'^J K^' Siven a clue; in fact parcents correct 

were slightly lower. Not unexpectedly, 38% of the students got none of these four 
items correct. lucoc luul 
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Table 32 

Item Analysis 
CSMP Subtest 3 



Test Items (from Test C) ; Percent Correct 

Put <, = or > between each pair 



1. 


4x6 


6x4 


69 


2. 


5x22 


5x25 


90 


3. 


4x(3+2) 


4x5 


49 


h. 


1x12 

2 


1x12 

3 


63 


5. 


1x58 

2 


1x56 

2 


79 


6. 


1x27 


1x27 

2 


58 


7. 


1x240 


1x240 


40 



5 5 



Mean Score 4.A8 
Correlation with Pretest .61 
KR20 .40 



The items in Table 32 tested what might be called basic understanding of the 
process of multiplication; no computation was required. The last four Itcas 
should be reviewed in the context of <.able 6, which deals with salcnlattng balveii 
and thirds of whole numbers. Student performance seems adequate, though one might have 
thought that more students would have known that | of 12 is greater than \ of 
12. The relatively low vcdue of KR20, indicating that the test items were not 
particularly homogeneous, may be due to the two kinds of items involving whole 
numbers and fractions, and to the element of guessing. 
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Table 33 



Item Analysis 
CSMP Subtest 4 



Test Icems (from Test C) 


rercenu Lorrecc 


1. 


60 


4- ^n 


A . 

- 80 












A 




Circle the true number sentence 


62 




60 


A 

+ 7n 


« fln 








2. 


25 

2; 


A 

+ 35 
A 

+ 35 


- 60^ 
A 

- 10 










25 


A 

A 

+ 35 


« iO 


Circle the true number sentence 


42 




25 












3. 


8 


A 

+ 6 ■ 








71 


A 




A 








52 


5. 


A 
5 


A 

+ 4 . 








CO 




A 


-4- 1 > 








42 


7. 


A 
5 


+ 5 • 








Aft 


a 


A 
1 S 


A 








57 


9. 




+ 10 ■ 








A1 


10. 


u 


. ^ . 


C®o®o®n 


i) 


ei 
















11. 


A 

5 


+ 11 


■ ( 






74 
















12. 








49 


13. 


A 

5 


+ 10 








79 


14. 




A 

+ 16 








53 












Mean Score 


8.43 












Correlation with Pretest 


.63 












XK20 


.87 



The Items in Table 33 deal with adding integers, including negative 
numbers . 

A perusal of Table 33 incMcate?: that ail four of the it:eins with a positive 
ansver (items 3, 9, 11 and 14) were rjore frequently correct than all sev2n of 
che items with a negative answer (items 4-6, 8, 10, 12 and 14). Vihen clues were 
given, exactly as shown, for the page containing items 10-14 performance improved. 
With the^addition "4+7=" percent correct rose from 52% (item 4) to 81% (item 10) 
and on "5+10" from 61% (item 9) to 79% (item 13). Student responses indicate that 
about half of the incorrect answers could be^accounted for by one of three 
explanations; a) the answer was reversed (8+6 would be 2), b) a "bat" was ignored 
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(8+6 would be 14), or c) a '^hat'* was added (8+6 would be 14). These three errors 
occurred about equally often. 

Integers are not taught in other second grade curricula so there is no basis 
for comparison of performance. There was only a slight improvement from first 
grade. For example 61% of last year's first graders got 3+8 and 67% of this 
year's second graders (generally the same students) got 5+S. Last year 44Z got 
2+3 and this year 52% got 4+?. Last year 61% got 9+T and this year 71% got 8+'^. 



Table 34 



Item Analysis 
CSMP Subtest 5 



1 


Test Items (from Test C) 


Percent Correct 


Put 


<, = or > between each pair 


Percent 


1. 


5 




76 


2. 


A 
12 


0 


51 


3. 


A 
6 




35 


4. 


A 

12+8 


A 

14+8 


32 


5. 


A 

58+49 


A 

58+45 


66 


6. 


A 

63+45 


A 

63+45 


51 






Mean Score 








Correlation with Pretest 


.37 






KJ120 


.52 



Students had the most difficnlry with items 3 and 4, These were the only 
two items in whicli the two numbers to be compared were both negative, Pji^obably 
many students compared the absolute values and obtained, for example, 8<6 ('>tri V) , 
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Table 35 



Item Analysis 
CSMP Subtest 6 



Test Items (from Test B) 



Percent Correct 



10. 



1. J_ X 6 = 



2. -i_ X 18 = 



3. J_ X 9 = 



X 48 = 



8. _L X 48 



A + A= 14 



X 14 = 



X 18 = 




7. J_ X 20 = 




40 + 8 




_L X 63 = 



X 254 = 



Mean Score 

Correlation with Pretest 
KI120 



94 



60 



81 



63 



81 



62 



57 



71 



33 



6.08 
.57 
.86 



This topic, multiplying by halves and thirds, also does not appear on standardized 
tests at this grade level. The performance of CSMP students is impressive. Items 5-7 
with one kind of clue app«ared on a later test page than did items 1-4. Thus one can ' 
compare performance on i xl8 from item 2 (wirhout clue) and item 6 (with clue). 
Evidently the clue did not help. Similarly one. can compare ix48 from item 4 and 
Item a and see only a moderate improvement, 

^5 0 
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Table 36 






Item Analysis 
CSMP Subtest 7 




TmC ICCBS 


(from Test A) 


Perccnc CorrecC 


Put numbers in Che boxes Co 
make Che senCences Crue 




I. 9 - 


□ - 6 


91 


2. L X 

2 


□ . 4 


67 


3. □ ^ 


5-7 


93 


4. 3 X 


C - 12 


64 




Mean Score 

CorrelaCion wiCh PreCest 
ICR20 


3.16 
.50 
.57 



The iteiris in Table 36 tested students' ability to work with open sentences and 
the results were quite good. Although items 2 and 4 were more difficult, as expected, 
one is impressed by the fact that two-tnirds of the students did get them correct. 
The moderate value of KR20 indicates that the items were not particularly homogeneous, 
probably because the various computational skills required were more important for 
success than any generalized understanding of how to work with open aentences. 





Table 37 






Item Analysis 
CSMP Subtest 8 




Test Items (from Test A) 


Percent Correct 


1. 


6 + (.; - 2) = 


60 


2. 


(2 + .i) K (3 - i) = 


59 




Mean Score 

Correlation with Pretest 
RK20 


1.19 
.44 
.64 



There was a:i error in printing item 2 from Table 3/, The middle sign 
should have been minus, yielding an answer of 4-2=?.. Tliv.^ item 1 would have 
been correct whether or not the students ignored thr, pa reu thesis but for item 2 
the student would have had to work first witViin ch'^ parenthesis. Otherwise hip 
answer of 2-r2-3-l would have been wrong, l/liat might ha/e beer, expected then 
would be a high percent correct on item 1 and some lower percent correct for 
item 2. The percent correct on the accual itc^ms was only moderate, however. 
One suspects, based on student interviews* that many students put down as their 
answer the contents on the parenthesis, rather than the complete expression. 
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Table 38 



Item Analysis 
CSMP Subtest 9 


Test Items (From Test A) 


Percent Correct 


Draw arrows for "I am more than you'* 




1. Arrov; from 9 to 7 

2. Arrow from 9 to k 

3. Arrow from 9 to 2 
4- Arrow froui 7 to 4 
5. Arrow from 4 to 2 


74 
84 
81 
81 
80 




Mean Score 

Correlation with Pretest 
KR20 


4,01 
.43 
.90 



The item in Table 38 was repeated from a test administered in first grade, 
to these same students. The mean number correct increased from 3.1 to 4.0 
during that time. The mean aui.:ber correct for the lowest scoring quarter 
increased from 0.95 to 1.35; nevertheless 15% of the students failed to get 
more than one of the 5 arrows drawn correctly. Sixty nine percent of the students 
got all arrows drawn correctly* 
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Table 39 



Item Analysis 
CSMP Subtest 10 






Test 


Items (from Test B)* 


1 Pp TP Pn f PnT Tfin ^ 








-13 






















/ 




>«167 






37i, 




■^"tV* >''148 








2X 




2X 








1. 


3 


X 37 = 


64 






2. 


_1_ 

n 


X 74 = 


37 






3. 


148 + 19 = 


bl 






4. 


4 


X 37 = 


29 




Mean Score 

Correlation with Pretest 
YRIO 


1.97 
.51 
.71 



*A different color was used for each different arrow. 



The diagram in Table 39 is an example of what is sometlmeo called a "Detective 
Story" in the CSMP curriculum. There is a mass of information given which was slightly 
less confusing with actual colors. Some answers are given explicitly as in tracking 
down the answers to items 1 and 3 by searching out the required information which is 
given directly for the student must realize that is 2x37-74 (shown) t^^^n - x74«37 . 
This is the idea of the "return arrow" ( ^ x) from 74 to 37. Similarly in item 4, 
a 2x arrow followed by a 2x arrow can le rv^placed by what we might call the 
composite 4x arrow. It is difficult to evaluate the adequacy of performance on 
this subtest. Although students do gee considerable practice with return arrows and 
ideas of composition, there are rather infrequent lessons dealing with diagrams as 
"packed" as this one and they are clearly intended as difficult problems. About a 
quarter of the :Jtudents did not get any items correct and it may be that for many 
of them the amount of information was simply overwhelming. 
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Table 40 

Item Analysis 
CSMP Subtests 11-13 



TMt Itw (Label dots and arxowa) 



1. - 4. Subcaac U (froit Taat A)* 
^*'^*0^co\xld be for ? 



5. - 8. Subtest 12 (from Test B)* 



4 could be tor " 



- 12. Subtest 13 (from Teat C)* 
+5..).. 8^ 



• could be for ? 



BAnga of Maaa Scores 

langc of Correlations th Pretest 

Enftt of 20's 



P«rc«nt Correct 



64 

62 

52 
33 



72 

56 

58 
51 



51 

54 

53 
36 



1.95 - 2.04 
.50 - .61 
.79 - .83 



*Diffar«nt colors vers used for ^>Ach different arrov. 



Subtests 11-13 were Identically formatted problems involving different 
operations. Three parts of each problem dealt with labelling dots on a fairly 
straightforward diagram. The computations were not difficult; for items 1-3, 
5, 6 and 10 there were similar or identical computations elsewhere in the CSMP 
or Standardized tests. (For example the computation 3x3 is required for item 2 
and that computation also appears in Subtest 1.) For these six items, the mean 
percent correct whj|n the item was posed in standard form without arrows was 
79, and for nhe same items in the context of an arrow diagram the percent correct 
was only 62. Thus there appears to be a significant number of students who were 
imable to figure our what to do - even though the diagrams aro not particularly 
complicated (they W3re much simpler looking in color) . For each of the three 
problems nearly a third of the students did not get any answers correct. 

There were two ways of determining what the answers to the last question of 
each section, what the answers could be for. One was to look at the arrow as a 
return arrow; knowing what the dots are for is unnecessary. The other was to look 
at the dots to determine what arrow could work between them, thus disregarding the 
opposite arrow. In the first problem, correct answers were about evenly divided 
between 2x (probably the first method) and +2 (probably the second method) and 29% 
of the students gave the incorrect answer of Ax, evidently a composition of two 
2x arrows. In the second problem, correct answers were again fairly evenly divided 
between I X And -9, and 28Z of the students gave for their incorrect answer the 

Er|c 5.. 
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coniposition of two «. x arrow*^. i.e., -^x. In the third problem almost all correct 
answers were +2, expected ul ier either method of solution, but ther ^re no 
particularly frequent inr.crrect errors. In fact there was a bewilc \g array 
of them. I 

4 

Table 41 

Item Analysis 
CSMP Subtest 14 



2. 



Test Item (from Test B) : 



Label the arrows 




Mean Score 

Correlation with Pretest 
KR20 



Percent Correct 



(Sign omitted) 



43 (19) 



37 (16) 



3 (6) 



0.83 (.41) 
.46 
.60 



The items in Table 41 dealt exclusively with composite arrows. Students were 
told that they could label dots to help them if they wanted to. Forty percent of the 
students did this, usually for all three items. The figures in parenthesis are the 
percent of students, in addition to those vho got the correct answer, who left the 
operation off their answer. (In the fir item this would give an answer of "10" 
instead of "+iO".) They were not counted as correct since 10 is also the label for 
the last dot if one starts from what might be a natural starting point of zero. 
This method works for the first two items (from 0 to 10 is +10 and from 0 to 5 is +5) 
but would yield an answer of •(+?) 0 (from 0 to 0) or just plain confusion for item 
3. These were difficult items; over half the students got none of the three correct. 

r 
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Table A2 

Item Analysis 
CSMP Subtest 15 (from Test D) 



Match these numbers to these dots 

(Dots were not numbered; see 
below for interpretation) 


Percent 
All L'orrect 


1 5. .16^ \ (79) 
\ .IS \ (55) J>f 

.!« .3 / (84)^^ y ^ 
^ (53) X X 

b) V ' A /\ 

\. 8 } / \ ^ ^ ^^^^ ^^^^ 
* ^ <75) ^ X (68) > 

c) I 1 m)yr V 

\ S / (59) JT^ . 

\. '5 .15 / '>r^' +1 


48 

64 
51 




Range of Correlations with Pretest 
Range of KR20's 


.21 - .43 
.83 - .88 



These problens were based on a format being tried out by the developers of the 
curriculum. Thus these students had not seen this type of problem before and con- 
siderable explanation, wlrh examples and individual assistance, v/as required. The 
idea in each problem '-as to use each of the numbers given by the string on the left 
exactly once on the dots in the diagram to the right., The dots were rot labelled - 
only the code for the arrow was given. Thus, the problem had a unique solution and 
there was much backtracking and figuring out exactly how one should go about starting 
the problem. The numbers in parenthesis are the number who got the right number for 
the dot, in the sense that the number was consistent with the numbers given on 
adjacent dot:(s) according to the arrow joining them. The exception in items 1 and 3 
was the left most dot of the longer path. That had to be the correct answer for the 
problem to be solved. Thus it can be seen that almost all students who got the 
correct number for chat dc t also got all the other dots labelled. The most common 
error was to start at the left most dot with the wrong "low number". In item a), 
either the long path would be 3, 5, 10 (wrong and the percent correct was lower), 
12, 14 and the short one 16, 18; or else 3, 5, 14 (v;rong again), 16, 18 and 10, 12. 
Thus the percent correct for the first and third dot in the long path is distinctly 
lower than for the remaining dots, which are all about the same. The same pattern 
is apparent in item c) , where students would start the long path with 3, 5 then 
get stuck and finish with 13, 15 or 10, 12. 
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Table 43 



Item Analysis 
CSMP Subtest 16 



Which dot is for 83 (from Test D) 



a) 




+2 



Percent Correct 



57 



b) 




+10 



52 



Mean Score 

Correlation with Pretest 
KR20 



1.10 
.54 
• 69 



The format of these r-^o problems was also unfamiliar to students. Each problem 
was solved by just over hc^lf the students. Students were told they could label as 
many dots as they wanted to, but that all they needed to do was to find and label 
the dot which was for 83, It is not possible to summarize the wide variety of 
responses for these items. More than half of the students who got the items correct 
labelled only one dot, 83, evidently figuring out the problem mentally. About 15 * 
percent of the students labelled five or more dots Incorrectly. 
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Test 44 



Item Analysis 
CSMP Subtt- L 17 



Test Items (from Test D) 


Percent Correct 


The arrows are for "I sent a card to you" 

Mary 

Tom Sue Joe ^ Bill 

1. Who got the most cards? 

2. Who did not get any cards? 

3. Who got exactly 2 cards? 

4. Wlio sent the nost cards? 


78 
73 
67 
55 




Mean Score 

Correlation with Pretest 
KR20 


2.73 
.52 
.77 



The problem in Table 44 was to interpret a non-numeric arrow diagram. The 
reader may verify that it is surprisingly easy to make an error on any of these 
items. The decreasing percent correct may be partially explained by a regular 
increase in the percent of students who did not put down any answer at all 
(5%y 8%, 12%, then 14%), this being the last subtest of Test D. On only one 
item did more than nine percent of che students answer any particular wrong 
answer. On item 4, 14 percent of the students said Bill sent the most cards; 
in fact he or.ly tied with Joe for second most. Twelve percent dl: the students 
got none of the four items right and 45 percent got all four correct. 
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Minicomputer Test (Test E) 

This test was administered individually to students in vine classes (usually 
six per class). The teiiter took the student through the series of Minicomputer 
tasks given below. 



1. What number is shown on the Minicomputer? 

(a) Does the student know enough to make a forward play 
(i.e., to reduce to a standard configuration)? 

(b) Does the student make the backwards 40-20+20 move, 
in order to play 80+20=100? 

(c) Does the student make tl'.e backwards 4=2+2 move 
in order to play 8+2=10? 

(d) (If not - remove two ones checkers). 
Does the student make 8+2=10 move? 

(e) Does the student read answer correctly? 



0 «• 



2. 427+218 



(a) Does the student set up Minicomputer correctly? 

(b) Does the student begin making forward plays? 



3. 2x305 



(a) Does the student set up Minicomputer correctly? 

(b) Does the student begin making forward plays? 

(c) Can he read answer correctly? 



4. 64-9 • 

(a) Does the student set up Minicomputer correctly? 

(b) Does the student begin to make backward moves? 

(c) Does the student cancel at least one pair of checkers? 

(d) Does the student make necessary 10=8+2 backwards move? 

(e) Does the student get correct answer? 

(f) Does the student read answer correctly? 



5. 768-427 

(a) Does the student set up Minicomputer correctly? 

(b) Does the student begin to make backward moves? 

(c) Does the student cancel at least one pair of checkers? 

(d) Does the student get correct answer? 

(e) Does the student read answer correctly? 

6. 1x320 



(a) 


Does 


the 


student 


(b) 


Does 


the 


student 


(c) 


Does 


the 


student 


(d) 


Does 


the 


student 


(e) 


Does 


the 


student 
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7. 3x24 



(a) Does che student set up Minicomputer correctly? 

(b) Does the student start to make plays? 



8. 7x15 



(a) Does the student set up Minicomputer correctly? 

(b) Does the student make a backward move to get pair of checkers? 

(c) Does the student make particular backwards move 10=8+2? 

(d) Does the student get correct answer? 

(e) Does the student read answer correctly? 



9. Tx2I 



(a) Does the student set up Minicomputer correctly'i 

(b) Does the student inake backward move 20=10+10? 

(c) Does the student get correct answer? 

10. Given the following configuration: 



Does the number get larger, smaller or still the same if this is done? 

(a) Move checker on 80 to 40. 

(b) Take two checkers off 100 and put one checker on 200. 

(c) Move checker on 2 to 20. 

(a) - c) were demonstrated to student without comment.) 



Considerable leeway was taken in administering and coding the test. The 
emphasis was not on whether the student could get the correct answer but on whether 
he knew how to go about doing the problem and whether he had certain rather general 
"Minicomputer Skills." Students were not necessarily given every item. For 
example when the student was clearly unable to do basic things he was not given the 
later more difficult problems or if the student had already, on item 5, demonstrated 
skill in subtracting on the Minicomputer then item 6 was omitted. Occasionally the 
tester was uncertain about the student's understanding and certain kinds of questions 
might be repeated. 

A set of 12 Minicomputer skills was considered and a determination made as to 
whether or not the student possessed this skill (or knowledge as the case may be). 
That is, a binary (Yes - No) decision was made. The student was given the benefit 
of the doubt; for example if he once demonstrated how to set up the Minicomputer 
and correctly began to subtract, subsequent errors would not change the positive 
decision that he did know how to subtract on the Minicomputer. In most cases this 
decision w^s relatively easy to make - it was apparent to the tester. For some of 
the skills (9, 10, 12 below) it is possible, though unlikely, that the student 
might do much better or worse on another day. Skill 5 was difficult to judge, 
primarily because of guessing, and probably has low reliability. In parenthesis 
after each skill is given the percent of students who were judged to possess that 
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skill. The skills are given in order of atcg^rved difficulty. 

1. Read numbers shown in standard form on the Minicomputer (81%) 

2. Show given numbers on the Minicomputer (81%)' 

3. Make simple forward plays on the Minicomputer (79%) 

4. Knows how to add nurribers on the Minicomputer (77%) 

5. Conservation on the Minicomputer (at least two correct from //lO 
above) (70%) 

6. Knows to make "special" forward play (8-f2=10) when simplifying 
configuration in adding or multiplying (68%) 

7. Knows how to multiply numbers on the Minicomputer (49%) 

8. Knows how to subtract on the Minicomputer (49%) 

9. Knows to make "special" backwards play to ones board (10=^8-f2) 
when required for subtraction or division (45%) 

10. Knows to play 4=2-f2 in order to produce necessary 8-f2 configuration 
when simplifying on MinicoiLputer (40%) ^ ^ 

11. Knows how to use Minicomputer to find 7, T of a number (30%) 

12. Knows to make special backward plays to tens board (100=80+20) 
when required for subtraction or division (23%) 

Basically the list collapses to tnree categories: reading and writing on 
the Minicomputer (1 and ,2) , how to set up and begin to solve an arithmetic operation 
(4,7,8,11), and techniques necessary to complete the solution (3,6,9,11). The 

test (using these 12 skills to generate a test score) was very homogeneous (KR=.91) 

and highly correlated with the pretest (r=.68). With the exception of skill 5, the 
skills were highly ordered. For instance 15 students did not have all four of 
the first four skills, and only five of these 15 had civon one (none more than one) 
of skills 6-12. Of 28 students who did not ger at least seven of the first 

eight skills (again excluding skill 5), only thr^^fi got skill 10 and none got skills 

11 and 12. 

There was a wide diversity of scores. Low ability students (as measured by 
pretest scores) did very poorly. Of the eleven students with the lowest pretest . 
scores (about 20% of the students tested), only two could both r6:ad and show 
numbers on the Minicomputer and only one of the eleven had more t:han four of the 

12 skills. On the other hand 10 students were correct on everything and another 
seven missed at most two skills. The 53 students who were tested on the Minicomputer 
were selected to representative of local second grade CSMP students and all have 
been in the program since the beginning of the first grade. 
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Further Analysis of Test Data 

Consistency of Results Across School Districts 

In order to determine the degree of uniformity of these results across the four 
participating school districts, mean scores acrosrn classes were calculated for each 
district. This Information is given in Table 45. Since different classes often 
wrote different tests (see the Testing Plan, p. 7) tl;e total score given in the 
table Is not based on any single group of students. It can be seen that fairly 
similar results were obtained In each district . However, these scores do rot take 
Into account differences In entering ability. Table 46 presents the same data, 
except that prior to calculating niean s\:ores by district the mean score for each 
class was adjusted by the usual procedure* to take these differences into account. 
With this adjustment the results are seen to be quite consistent, particularly in 
the comparison of CSMP - Non-CSMP performance, across the various districts. Thus 
It seems unlikely that results found previously significant can be attributed to 
particularly favorable conditions ±i\ r,nly one or two of the four districts. 

Table 45 
Mean Scores By District 





Dlitrlct 1 


District 2 


District 3 


District 4 




CSKP 


Non-CSMP 


CSMP 


Non-CSMP 


CSMP 


Non-CSMP 


CSMP 


Non-CSMF 


Coaparlson Teucs 

I 
II 
III 
IV 


37.8 
26.4 
21.2 


16.6 
31.2 
26.3 
20.2 


20.2 
46.1 
27.5 
22.2 


18.9 

42.6 
25.7 
20.7 


■ 21.2 
41.3 
25 . 7 
70.7 


16.9 
36,4 
25.3 

if;. 2 


U.l 
32.3 
22.5 
18.9 


12.0 
34.4 
23.6 
17.0 


Standfirdlzed Tests 


















Confutation 
Concepts 


20.1 
18.6 


19.4 
18.3 


23.2 
20.0 


21.7 
19.8 


22.2 
20.2 


20.4 
17.6 


19.5 
16.2 


18.4 
15.8 


TotiiX of Above 


U3.5 


132.0 


159.2 


149.4 


151.'' 


134.8 


123.5 


121.2 



Table 46 

Adjusted* Mean Scores by District 



Tttati 


District 1 


District 2 


District 3 


District 4 




















CSM? 


Nov-CSMP 


CSMP 


Non-CSMP 


CSMP 


Non-CSMP 


CSMP 


Non-CSMP 


Cooparisan Tests 


















I 


16.9 


15.3 


18.6 


16.3 


19.2 


16.5 


21.7 


21.9 


II 


39.2 


33.5 


41.2 


38.5 


40.0 


36.8 


37.5 


37.3 


. iir 


26.8 


25.5 


27.8 


27.2 


25.0 


25.6 


21.2 


22.8 


IV 


?.2.2 


10 . 7 


21.7 


19. A 


20.8 


18.5 


19.7 


17.5 


Stan<Uraized Tests 


















Coapuca;.lon 


19.8 


19.1 


21.4 


19.0 


20.7 


20.3 


24.2 


23.4 


CoQcaptB 


18.0 


W.9 


17.0 


15.6 


17.8 


17.6 


23.8 


23.2 


j 

! Total oS. Above 


142.9 


132.0 


147.7 


136.0 


143.5 


135.6 


148.1 


144.1 



*Adjusted class mean - class mean - r(clas3 pretest score - mean pretest 
score), where r is t.he correlation between test and pretest scores. 
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Students Wno transfer Into CSMP 

An important quest.iou about the CSMP curriculum is the degree to v;hlch students 
can successfully transfer Into and out of the CSMP program from a traditional math 
program. It has not been possible to track individual students who leave the program; 
however, the very strong pe^ f m-nance of CSMP studen*:s on the standardizt^id tests is 
solid evidence that thes^^ students would be well prepared, cognitive].y , for entering 
a traditional m-nth program. 

The problem for stucents entering the program, however ^ is much different. The 
languages of the Minicomputer, of arrow diagrams and of striug pictures do not occur 
in traditional programs and so clearly these students have some catchlny, up to do. 
Some students transferred into a second grade CSMP class at or near tr.e beginning of 
the year and it has been possible to investigate the performance of sir. a students. 
Their test scores were analyzed separately, as were the scores for students who were 
in the program from before the middle of first ^^rade. In addition to computing mean 
test and pretest scores, a regression equation was developed for each group so that 
mean test scores for student? of various ability groups could be estimated. 
Comparison Tests III and IV had a very small number of 'late enterers" and this 
analysis was not done for them. 

Table 47 

Test Scores By Time of Entry: 
(l)Before Feb. of First Gi«:de (2)During Sept. or Oct. of Second Grade 





Nusiber of 
Students 


Mean Test 1 
Scores 


1««n PrctssC 

Scnrts 


EstiMt«<l 
Indicated 

40 


Teat Score for 
Pretest Scores* 

50 60 


Comp«rison Teat I Entryd) } 
tncry(2) | 


85 
24 


19.4 
. 17.1 


50.1 
46.6 


15.5 


I 

19.4 




CoaparlBon Test 2 Entryd) 
, EntrvC2) 


111 
26 


38.1 
36.6 


53.0 

46.9 j 


?0.4 
31.1 


36.3 
37.3 




CTBS; EntryC;) 
Cocipviti.tion Encvyd) 


198 
50 


22.2 
20. B 


51.6 
48.1 


16.2 
17.7 


21.0 
20.7 


23.7 
23.7 


CT3S: Entry (U ] 
Concepts and Entry (2) 
Applications 


203 
51 


ig.4 

17.8 


51.9 
46.2 


16.4 
15.5 


16.9 
IB. 3 


21.5 
21.1 


Total of Above Tests 

E'.tryH) 
Entry (2) 






60.5 
79.0 


95.6 
94.5 


110.7 
110.0 


CSMP Ttist A Entry ( :) 
Ent ry (D 


17 


17.9 
U.2 


53.2 
44.2 


13.4 
12.9 


16.6 
16.1 


20.2 

19.: 


CSMP Te»c B Ent ry (1 ) 

Entry (2) 


83 
21 


•10.4 
9.2 


51.5 
53.0 


7.1 
4.9 


10.0 
6.2 


12.8 
19.2 


CSMP Test C Entryd) 
Entry(2) 


102 

27 


18.6 
13.4 


51.9 
53.0 


14.3 
11.2 


17.9 
14.5 


21.5 
17.9 


CSMP Taat D F.ntryd ) 
Entry (2) 


103 
31 


14.5 
10.5 


51.1 
49.8 


11.5 
7.4 


14.2 
10.5 


16.0 
13.6 


Totml of CSM? Testa 

Entryd) 
Entry (2) 




46.3 
36.4 


58.9 
49.3 


71.3 
62.3 



•Obtaln.d .ub.tltotlQg «0. 50 .nd 60 r..p.ctlv.ly for x In th. .<,u.ttoo / -T^\iix-f) 
wh«r. S mad y .r. th« group amma .cor*, for pr.t.at and tmat r«.p«ctlv.ljc «7 
cha corr.spondlat .Cniitd d«vl«tloo» md r 1. th« corr.latloo co«fflcl«Bt. 
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Tha first four ;ests listed in Table A7 are tests written by both CSMP and 
Non-CSMP students and do not require the concepts and skills unique to CSMP* The 
group of students who transferred into the classes usually had lower pretest scores 
and lower test scores. However, when regression estimates were made for students 
with mean pretest scores of 40, 50 and 60 (roughly the 20th, 50th and 75th 
percentile among all the students who were pretested), it is found that whether 
the student joined their class in first grade or in second grade made very little 
difference in his scores. The data suggests that CSMP students are not penalized 
in the acquisition of standard content by transferring into a CSMP class. The 
largest difference was in the Comparison Test 1^ late-enterers scoring about a 
point below the other students. 

The CSMP tests present a different picture. There are much larger differences; 
late-enterers scored at least two and as much as four points lower at all ability 
levels on Tests C and D and at the lower ability level for 'Test B. Vu^.le the 
differences existed on almost all the individual subtests, they were largest with 
those dealing with integer calculations (Subtest 4) , within certain arrow diagram 
problems (Subtests 10, 11 and 15), and with choosing the larger answer given two 
fractional computations (parts of Subtest 3). The items of these various subtests 
are shown in Tables 33, 39, 40, 42 and 32 respectively. 

As previously noted, because of the limited number of students who could be 
tested individually, only students who had been in CSMP since first grade were 
tested wlch the Minicomputer. Hence it is not known whether late-enterers caught 
up with other students in this area. It should also be noted that the data 
provide no answers whatsoever about the relative progress of students where an 
entire class begins to use the CSMP class for the first time in the second grade. 

Performance By Ability Le/els 

All CSMP students who had a mean pretest score below 40 were combined into one 
group and mean pretest and test scorc-.s computed for this group. This was also done 
for students with a mean pretest score cf over 60. The same thing was done for 
Non-CSMP students. One can then separately compare CSMP and Non-CSMP performance 
for low ability and then for high ability students. For example, CSMP students 
had higher scores on the standardized tests; were they higher at only one level of 
ability or '^across Che board'*. Ti»e data ai'e presented in Table 48. 
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Table 48 



Test Scores by Ability Groups* 



Test 


Farcentage of Students 


Mean Pretest Score 


Ke«n Test Score 


Low 


Hlfeh 


Low 


High 


Lou 


High 


Comparison Test I CSMP 


23 


15 


3U3 


67.7 


14.6 


27.9 


Non-CSMP 


23 


23 


29.5 


65.9 


12.7 


21.2 


Comparison Test II CSMP 


16 


30 


28.2 


66.6 


21.4 


45.9 


Noa-CSMP 


10 


21 


34.2 


'67.7 


28.3 


43.6 


Comparison Test III CSMP 


U 


27 


33.0 


67.2 


. 22.7 


29.6 


Nou-CSMP ' 


9 


27 


36-6 


67.5 


24.0 


27.6 


Comparison Test IV CSMP 


8 


X9 


34.9 


66.2 


12.1 


26.7 


Non-CSMP 


5 


3'/ 


34.0 


67.5 


10.8 


24.5 


CTBS : CSMP 


19 


lU 


30.1 


66.5 


IA.5 


25.6 


Computation Non-CSMP 


16 


Ik 


30.3 


66.8 


13.5 


25.1 


CTBS: Concepts & CSMP 


18 


2U 


30.0 


66.6 


13.1 


22.9 


Applications Non-CSMP 


16 




30.4 


66.8 


11.9 


22.4 



*Lov Ability Group: Students with pretest score <40 
High Ability Group; Students with pretest score >60 



The answer to the question raised above is that for the standardized tests, 
both high and low ability CSMP students scored higher than the appropriate 
Non-CSMP students, though the difference was larger for low ability students. 
Generally at both levels of ability, CSMP students outperformed the Non-CSMP 
students. The exceptions are for low ability students on Comparison Tests II 
and III where the Non-CSMP students did better, though in those cases the low 
group for CSMP was distinctly lower than for Non-CSMP as shown by the fairly 
large differences in pretest scores. 
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Summary and Discussion 

Thirteen second grade classes In four districts In the local area studied 
the CSMP curriculum during the 1974-75 school year. A wide range of abilities 
and socio-economic status was represented by these classes (see Tables 1 and 5). 
For each CSMP class there was a comparison class, In the same school where 
possible ^md In an adjacent school where the CSMP class was the only second grade 
class. Tnese classes were usually the same grcups of students who were tested 
the year before In first grade In the CSMP - Non-CSMP comparisons described In a 
previous report.* In the fall a test of mental abilities was administered to all 
students and the scores from this test were used as covariates in the subsequent 
analysis of test data in order to adjust for possible differences in entering 
ability between CSMP and Non-CSMP classes. Because of the rather reful pairing 
of classes such dlf f erencvViS were generally quite small. 

Two kinds of tests, a standardized test and a series of comparison tests, 
were administered to CSMP and Non-CSMP classes according to a testing plan (Table 6) 
which ensured that representative pairs of classes would write each test. In the 
case of the standardized test all 13 pairs of classes were used. The standardized 
test used was the Mathematics Test, with two subtests, of the Comprehensive Test 
of Basic Skills. Four Comparison Tests with varying numbers of subtests were 
developed, two group-administered to the whole class and two administered to about 
seven representative students of each selected class. These comparison tasks were 
essentially problems posed in situations which were novel to both CSMP and Non-CSMP 
students. They were Intended to be situations in which the techniques and ways of 
thinking about mathematics which are stressed in the CSMP curriculum might be used 
with more success than would normally be the case. 

Table 49 summarizes the results of these tests. Mean scores across classes 
are given for the CSMP and for the Non-CSMP classes and those tests on which the 
differences were significant at the .05 level are Indicated. A two-way Analysis 
of Covariance was used, with classes as the unit of analysis, and the resulting 
F-test had between 1 and 4, and 1 and 11 degrees of freedom. 



EKLC 



*Evaluatlon Report l-A-3: Final Summary Report Vgar 1 



60 



Table 49 



Summary Data for Tesrs 
Administered to CSM? and Non-CSMP Classes 





Muaber of Pairs 


1 

Mean for j 


Mean for 


Significant 


Test 


of Classes 


CSMP Classes 


Hon-CSMP Classes 


at .05 level 


Comparison Test I 










a) Word Problems 


1 


4.79 


4.06 




b) Number Puzzles 




2.66 


2.51 




c) Estimation: Calculation 


6 


5.18 


5.n 




d) Estimation: Largest Number 






A A A 


7 

V 


e) Showing Fractions 




1.15 


0.92 




Total 




19.06 


17.01 




Comparison Test II 








/ 


a) Equations: Construction 




2.72 


2.03 




7 


13.63 


11.19 


/ 


c) Combinatorics 




16.94 


17.08 




d) Regrouping 




6.34 


6.55 


/ 


Total 




39.62 


38.86 


Comparison Test III 










a) Classification 


12 


17.98 


17.23 




b) Binary Relations 




8.05 


8.12 




Total 




26.03 


25.36 




Comparison lesc IV 










a) Nmber Patterns 


11 . 


10.11 


•9,28 


/ 


b) Functions 




6.48 


5.75 




c) Probability 




4.59 


3.99 


/ 


Total 




21.18 


19.01 


/ 


Comprehensive Test of 










Basic Skills 








/ 


a) Computation 


13 


21.63 


20.28 


b) Concepts and Applications 




18.96 


18.12 


/ 


Total 




40.61 


38,47 


/ 



All differences which were significant were in favor of CSMP. These included 
both the Computation and Concepts parts of the standardized tests and several of 
the various Comparison Tests. Wich the exception of Comparison Test IVc) ^ 
Probability, the four subtests on which CSMP classes did significantly better than 
Non^CSMP classes were what might be called "directly numerical" situations; numbers 
(or numerals) themselves were the stimuli. Other subtests in which the given 
stimuli were situations in which numbers (or more widely, mathematical thinking) 
were to be applied (la, le, lie, Ild, Ilia, Illb) did not produce significant 
differences . 

These results are very encouraging. CSMP students did better on both parts 
of the standardized test and on several novel tasks of a numerical nature. It is 
also important to note that the results were consistent across school districts 
(see Table 46) and, perhaps more importantly, across ability levels. It is not 



61 



the case that differences were due to gains by high ability students; loxv? ability 
students also did better (see Table 47). In addition, rtudents who transferred 
into CSMP at the beginning of second grade appeared to do as well as students who 
were in CSMP from first grade except on Comparison Test I where they had slightly 
lower scores (Table 48). 

In the author's opinion there are two explanations worth considering other 
than the obvious one that the CSMP curriculuin is responsible for these differences.^ 
The first explanation is that teachers were not randomly assigned to CSMP and 
Kon-CSMP (indeed this is virtually impossible to accomplish in the pilot stages 
of any program implementation). It is therefore possible .that CSMP teachers may 
be, as a group, more capable of producing student achievement. This is a plausible 
explanation and cannot be entirely refuted. However, in discussions with principals 
who had both a CSMP and a Non-CSMP teacher in their school, there was no support for 
such differences; indeed in one case the same teacher taught both classes and in 
another the principal thought the Non-CSMP teacher was a better teacher. It is also 
true that this selection factor would be stronger in the first year of this study 
v;hen the very first teachers to use CSMP are chosen. The next year, the second grade 
teachers more or less inherit the program and the classes. 

The other explanation may be that CSMP classes spend more time in math 
instruction than do Non-CSMP classes. Based on interview cota*, the mean number 
of minutes per day on math instruction in the 13 CSMP classes was 50, which is 
slightly higher than usual. However, there was no relationship between number of 
minutes and achievement either with or without adjusting for differences in entering 
ability. This information is not available for Non-CSMP classes though it is known 
that, in at least some classes, school scheduling dictated that the same time be 
spent for all math instructions. 

Thus, while these two alternatives cannot be ruled out, there are good reasons 
for discounting them as explanations for higher achievement by CSMP students. The 
so-called "novelty effect*' is also discounted for two reasons. First, described in 
the report dealing with student interviews'^*, students were usually not aware that 
their math program was particularly different from what other students used. Second, 
data collected over the past two years from first grade students indicates that, as 
judged by student performance, teachers do as v.'ell if not better in their second year 
of teaching CSMP as they did the first year.^'^'-^ 

One is led to believe from wluit is presented above that CSMP students did 
,:>i^T if icantly better than Non-CSMP students on several measures and that this can 
be attributed to the CSMP program and not to other factors. The reader should bear 
in mind that these results can not nece.ssfirily be generalized to school systems or 
■••^'•.sset, or teachers beyond those participa ting in this study (ie. "local classes"). 
That is one of ^hc penalties for n.^n- random (in this case voluntary) sampling of 
school districts, and within disL'r:cts, teachers. 



^Evaluation Report 2~C-3: Teacher Interviews^ First Grade 
**Evaluation' Report 2-6-3: Student Interviews 
***Evaluation Report 2-B-2: Readministration of First Grade Test Items 
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A series of tests dealing with the unique aspects of CSMP, content not already 
covered in one of the above tests, was also administered to CSMP classes. The 
results are more difficult to interpret because of the lack of standards, either 
through what "usually happens" (control groups) or what "should happen" (a 
priori standards of achievement laid down by the program). Thus the interpretatiovi 
of the data here is the author's own subjective evaluation of what one should expect 
second grade CSMP students to be able to do. However, this evaluation is b.osed on 
knowledge of curriculum, the contents of the various lessons and relative emphasis of 
topics; and on discussions with teachers^ observations of classes and interviews 
with students. 

Rather than reviewing and commenting on the results of each subtest, which has 
already been done, the discussion will center on general topics. ^ 

a) Multiplication. The level of achievement with simple, whole-number, 
single-digit, multiplication is quite satisfactory, as indicated also 
by the standardized test data. It is also very satisfactory in two 
areas not typically given much consideration in second grade. A 
surprising number of students were able to double or triple relatively 
large numbers; nearly half the students could calculate "2x37". And in 
calculating fractional parts of a whole number, specifically 1/nth of a 
number, students also did very well; about half the students could 
calculate "ix48" and "ix20." 

b) Integers. Negative integers are not studied in other seccnd grade 
curricula. Hence there is no basis of comparison and it mfy safely 
be said that: whatever progress CSMP students make is a "gain" over 
Non-CSMP students. The level of success is judged to be adequate; 
60% of the students could select the one true number sentence ot four 
very similar and possibly confusing sentences involving negative 
in^tegers (Table 33), though only about half the students could compute 
"A+7" and "5+4." However, the improvement from first grade is rather 
disappointing. There was only a moderate increase from last year in 
the percentage correct on those i'tems repeated from last year's test. 

c) Arrow Diagrams. The method of looking at percent correct is not entirely 
satisfactory for this topic. Generally these percentages are satisfactory. 
On the one hand there are surprisingly high percentages of students getting 
difficult items correct; for example the "Detective Story" items in Table 

39 and the problems in Table 42 whose format was completely new to students. 
On the other hand there were many subtests on which up to 20% of the students 
had virtually no success. For example, 

i) 12% of the students got none of the four "sending valentine" 
problems in Table 44 and 15% got no more than one of the five 
relatively easy "I am more than you" arrows in Table 38. 
ii) For each of the three subtests requiring dots and arrows to 
be labelled (Table 40), over 20% of the students got none cf 
the four test items correct, 
iii) In labelling dots in order to locate thie dot for 83 in Table 43, 
15% of the students made five or more errora. 

d) Minicomputer. Again there are a very wide range of scores; many could 

do virtually everything asked and many others could do virtually noc:hing. 
Over 30% of th«e students made at most one error on this rather comprehensive, 
individually administered tnst and these students generally worked very 
rapidly. On the other hand when considering low ability students (the 

(3^) 
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bottom 20% of the CSMP students as measured by the pretest), only two 
of these 11 students could consistently read and write numbers on the 
Minicomputer, let alone even set up the Minicomputer for subtraction, 
multiplication or taking one-half of a number. 

Students who transferred Into a second grade CSMP class during the first two 
months of school do not, as a group, perform as well on these tests of CSMP 
content as students who were in the program from the beginning (Table A7). 
Thus while new students do not appear to be penalized as far as the standard 
skills and concepts, they do not, during the course of the year, "catch up" wIl:; 
the other students in acquiriiig those concepts specific to CSMP. This is not a 
surprising result. 

In suannary then, CSMP classes did better than Non-CSMP classes on both 
standardized measures and some "content*-free" comparison tasks and this is 
true for both high and low ability students and ^or both original and transfer 
students* Although overall perform*.nce was generally adequate and at times 
praiseworthy for unique CSMP content, on those subtests involving arrow diagrams 
and the Minicomputer a considerable number of students had little or no success, 
and students who transferred into CSMP classes near the beginning of the year 
were not able to catch up with their classmates on CSMP concepts. 



7 (J 



ERIC 



64 



Appendix 
Analysis of Covariance Tables 

For each of the tests administered to both CSMP and Non-CSMP 
classes an analysis of covariance table, in rather standard 
form, is presented. To illustrate this format the first 
part of Table A* L is reproduced below, dealing with the 
addition items from the Computation Test of The Comprehensive 
Tests of Basic Skills. 



Test 


Source of 


Degrees of 


Sum of 


Mean Sum 


K-Rntio 




Varlat ion 


Freedom 


Squareti 


of Squares 






Li ror ( iiMui iiig reg revs:, i u:i) 


i: 


1.59 










1 


.38 


.58 


6.33* 


Addition Items 


Error(aftiT regression) 


11 


1 .01 


.09 






School s 


12 


4.02 


.3A 


... 




CShtr - Non-CSMP 


1 


3.31 




36.04* 



1. The first source of variation is the error, or residual, unexplained 
variance not accounted tor by the additive analysis-of-variance 
model wherein class score is predicted by a general mean plus an 
effect due to scnool (the school from which the CSMP - Non-CSMP 
pair was drawn) plus an effect due to curriculum (CSMP versus 
Non-CSMP). This model does not take into account regression on 

the covariate (pretest). 

2. The next term "Regression*' shows the reduction in this error variance 
when the model is expanded to take into account differences in the 
pretest scores of the various classes. A large reduction relative 

to the original error variance indicates that the expanded n-odel 
provides a much better interpretation of the data. In the above 
example, approximately one-third (.58 of 1.59) of the previously 
ur'ixplained error variance can in fact be explained by taking 
differences in pretest scores into account. 

3. T'he third term is the reduced error variance (the original error minus 
regression) and is the standard against which othor variances are 
judged for significance . 

4. The fourth term is the variation in class scores which can be explained 
by sy,stematic differences from school to school after fitting the 
expanded modnl. This is not of particular interest in this study, 
except to note that this factor often did indeed account for a 
considerable portion of tlic variance and was worth including in 

the model. 
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5. The fifth term is the variation in class scores which can 
be explained by differences in curriculum (i.e. between 
CSMP and Non-CSMP) after fitting the expanded model. 

Each Sum of Squares is divided by its number of degrees of 
freedom to obtain the Mean Sum of Squares, thus providing a sort 
of average of the amount of variation due to each source. The 
F-ratio for any particular source of variation is simply its 
Mean Sum of Squares divided by the Mean Sum of Squares due to 
unexplained error. If this ratio is large, i.e. if the systematic 
error due to that source of variation is much larger than due to 
random, unexplained error, then the result is significant. In the 
above example, the F-ratio for CSMP - Non-CSMP differences is 
3.31t.09=36.09 with 1 and 11 degrees of freedom. This is very 
large, the largest encountered in all the comparisons, and the 
probability that the differences were merely random errors is 
extremely small; i»e. the differences are significant. All 
F-ratios which are significant are indicated by an asterisk and 
any such significant differences in the CSHF - Non-CSMP com- 
parisons are in favor of CSMP classes. ' 
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Table A.l 



Analysis of Covariance: 
Comprehensive Test of Basic Skills 





Sourca of 


Degraaa of 


Siai of 


Haan Stm 


F-Batio 




Variation 


Praadon 


Squaraa 


of S<|aar«s 






Error (Ignoring regression) 


12 


1.59 






Adaltlon Iceins 


Regression 


I 


.58 


.58 


6.33* 


Error(after regression) 


II 


1. 01 


.09 






Schools 


12 


4.02 


.34 


3.65 




CSMP - Non-CSMP 


I 


3.31 


3.31 


36.09* 




Error (Ignoring regression) 


12 


3.70 








Ragresslon 


I 


.52 


.52 


1.78 


Subtracclon Icems 


Error(after regression) 


II 


3.18 


.29 






Schools 


12 


13.08 


1.09 


3.77 




CSMP - Non-CSMP 


I 


1.45 


1.45 


5.04* 




Error (Ignoring regression) 


12 


2.45 








Regression 


r 


.05 


.05 


.23 


Multiplication Items 


Error(after regression) 


II 


2.40 


.22 






Schools 


12 


6.57 


.55 


2.52 




CSMP - Non-CSMP 


I 


.28 


.28 


1.27 




Error (ignoring regression 


12 


9.62 








Regression 


I 


2.83 


2.83 


4.58 


Total Computation 


Error (after regression) 


II 


6.79 


.62 






Schools 


12 


41.04 


3.42 


5.54* 




CSMP - Non-CSMP 


I 


11.68 


11.68 


18.92* 




Error (Ignoring regression) 


12 


26.03 








Regression 


I 


19.14 


19.14 


30.57* 


Concepts & Applications 


Error(after regression) 


II 


6.89 


.63 






Schools 


12 


26.45 


2.20 


3.52* 




CSMP - Non-CSMP 

— 


I 


3.93 


3.93 


6.27* 




Error(lgnorlng regression) 


12 


58.85 








Regression 


I 


33.66 


33.66 


14.69* 


total Mathonatlcfl Tmt 


Error (aft sr regraaaioa) 


II 


25.19 


2.29 






$cbool« 


12 


104.73 


e.73 


3.81* 




CSMP - Non-CSMP 


I 


32.64 


32.64 


14.23* 



o 
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Table A* 2 



Analysis of Covariance 
Comparison Test I 





Test 


Source of 


Degrees of 


Sum of 


Mean Sum 


F-Ratlo 






Variation 


Freedom 


Squares 


of Squares 








Error ( ignoring regression) 


c 

J 


3.** / 










Regression 


1 


1.09 


1.09 


1,85 




Vord Problems 


Error(after regression) 


A 


2.37 


.59 








Schools 


5 


3.21 


.6A 


1.08 






CSMP - Non-CSMP 


1 


1.18 


1.18 


1.98 






Error ( ignoring regres sion) 














Regression 


1 


.12 


.12 


1.07 


lb) 


Ntonber Puzzles 


Error(after regression) 




.AA 


.11 








Schools 


5 


.21 


.OA 


.37 






CSMP - Non-CSMP 


1 


.OA 


.OA 


.39 






Error ( ignoring regression) 


5 


2 . 19 










Regression 


1 


.66 


.66 


1.75 


Ic) 


Estimation-Calculation 


Error(after regression) 




1.5A 


.38 








Schools 


5 


2.30 


.A6 


1.21 






CSMP - Non-CSMP 


1 


.00 


.00 


.01 






Error (ignoring regression 


5 


.9A 










Regression 


1 


.00 


.00 


.01 


Id) 


Estimation- Largest Number 


Error(after regression) 




.9A 


.23 






Schools 


5 


.7A 


.15 


.63 






CSMP - Non-CSMP 


1 


2.00 


2.00 


8.54* 






Errordgnorlng regression) 


5 


.26 










Regression 


1 


.06 


.06 


L.IO 


le) 


Showing Fractions 


Error(after regression) 




.20 


.05 


.2A 






Schools 


5 


.06 


.01 






CSMP - Non-CSMP 


1 


.lA 


.lA 


2.67 






Errordgnorlng regression) 


5 


16.01 










Regression 


1 


5.55 


5.55 


2.12 


I 


Total 


Error(after regression) 


A 


10. A6 


2.62 






Schools 


5 


13.06 


2.61 


1.00 






CSMP - Non-CSMP 


1 


10.02 


10.02 


,.»3 j 
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Table AO 



Analysis of Covariance 
Comparison Test II 



Test 


Variation 


Freedom 


Sun of 
Squares 


MftAn SuD 
of Squares 


F-Ratio 


Ila) 


Equations-Construction 


Error ( ignoring regression) 
Regression 

Error vafter regression) 

^ fhnn 1 Q 

CSMP - Non-CSM? 


6 
1 
5 
6 
1 


.36 
.02 
,33 
.11 
1.63 


,02 

.o: 

.02 
1,63 


.33 

.28* 
24.44* 


lib) 


Equations-Fluency 


Error (ignoring regression) 
Regression 

Error(after regression) 
Schools 

CSMP - Non-CSMT 


6 
1 
5 

c. 
0 


9.06 
2.29 
6.77 

J. U . 1 J 

19.18 


2.29 
1.35 

1 f>Q 
L . 07 

19.18 


1.69 

1 

14. 16* 


lie 

i 


r n J <' aatorics 


Error ( ignoring regression) 
Regression 

Error(after regression) 
Schools 

CSMP - Non-CSMP 


6 
1 
5 
6 

I 


b.43 
.99 

.77 
.14 


.99 
1.00 
.13 
,14 


.91 

.12 
.13 


Ad) 


fe^xTw' Ing 


Error (ignorint: regression) 
Regression 

Ertor(after regression) 
Schools 

CSMP - Non-CSMP 


6 
1 
5 
6 
1 


7.40 
';.29 
2.12 
4.39 
.39 


5.29 
.46 
,73 
.39 


f.0.98* 

1.58 
.85 


II 


Total 


£rror(ignoring regression 
1 -vegression 

r?.ror(after regression) 

S<:rool8 
1 - Von-CSMP 


f 
1 
5 
6 
1 


37.53 
23.60 
13.^3 
11.39 
2K09 


23.60 
2.79 
1.90 

21.09 

1 


8.47* 

.68 
1 7.57* 
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Table A* A 



Analysis of Covariance 
Comparison Test III 





tmmt 


Source of 


Degrees of 


Sum of 


Mean S um 


F-Ratio 






Variation 


Freedom 


Squares 


of Squares 




1 

i 




Error(ignoring regression 


11 


16. A6 






Ilia) 




Regression 


1 


.90 


.90 


.58 


Clap Ification 


Error(after regression) 


10 


15.56 


1.56 








Schools 


11 


35.50 


3.23 


2.07 






CSMP - Non-CSMP 


1 


1.3A 


1.3A 


.86 






Error (ignoring ^egression) 


11 


5.3A 






< 

11 It; 




Regression 


1 


1.36 


1.36 


3.A2 


.nnary Relations 


Error(after regression) 


10 


3.98 


AO 






Schools 


11 


22.05 


2.00 


5. OA* 






CSMP - Non-CSMP 


1 


.51 


.51 


1.29 






Error (ignoring regression) 


11 


17.82 










Regression 


1 




A.A7 


3.35 




Total 


Error(after regression) 


10 


13. 3A 


1.33 






Schools 


11 


95.27 


8.66 


6.A9* 


i 




CSMP - Non-CSMP 


1 


.19 


.19 


.lA 
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Table A- 5 



Analysis of Covariance 
Comparison Test IV 



Test 


Source of 


Degrees of 


Sum of 


Mean Sun 


F-Ratio 




Variation 


Freedom 


Squares 


of SqxMres 






ErrorCignoring regression) 


10 


7,3A 








Regression 


1 


,27 


.27 


,3A 


IVa) Number Patterns 


Error(after regression) 


9 


7.07 


,79 






Schools 


10 


19, 2A 


1,92 


2,A5 




CSMP - Non-CSMP 


1 


3.99 


3,99 


5.08* 




ErrorCignoring regression) 


10 


10.83 








Regression 


1 


1.71 


1.71 


1.69 


IVb) Functions 


Error(after regression) 


9 


9.12 


1.01 






Schools 


10 


16.06 


1.61 


1.59 




CSMP - Non-CSMP 


1 


3, AO 


3,40 


3,36 




ErrorCignoring regression) 


10 


3.86 








Regression 


1 


1.11 


i.U 


3.6A* 


IVc) Probability 


Error(after regression) 


9 


. 2.75 


,31 




Schools 


10 


8.25 


.62 


2.70 




CSMP - Non-CSMP 


1 


1,65 


1.65 


5. AO* 




ErrorCignoring regression) 


10 


26.25 








Regression 


1 


.60 


.60 


.21 


IV Total 


Error(after regression) 


9 


25.65 


2.85 






Schools 


10 


78.18 


7.82 


2.7A 




CSMP - Non-CSMP 


1 


26. A7 


26.47 


9.29* 
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